bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Alan Mackenzie]

Hello, Emacs.

In the following function:

(defun replay-parse-state-state ()
  (message
   (concat "(setq "
    (mapconcat
     (lambda (arg)
       (format "%s %s%s" (car arg)
	       (if (atom (cdr arg)) "" "'")  <==================
	       (if (markerp (cdr arg))
		   (format "(copy-marker %s)" (marker-position (cdr arg)))
		 (cdr arg))))
     c-parse-state-state "  ")
    ")")))

, the quote mark on the marked line is an ASCII quote, 0x27.  When this
function runs in the Emacs master branch under X-Windows, the output of
this quote gets corrupted into a curly quote, as follows:

                    |
                    v
(setq c-state-cache ’((32429 . 33731) 795 365)  c-state-cache-good-pos
33731  c-state-nonlit-pos-cache ’(53334 50334 47334 44334 41334 38334
35334 32334 29099 26099 23099 20099 17099 13993 10731 7641)
c-state-nonlit-pos-cache-limit 53334  c-state-semi-nonlit-pos-cache
’(32334 29099 26099 23099 20099 17099 13993 10731 7641)
c-state-semi-nonlit-pos-cache-limit 32334  c-state-brace-pair-desert nil
c-state-point-min 1  c-state-point-min-lit-type nil
c-state-point-min-lit-start nil  c-state-min-scan-pos 1
c-state-old-cpp-beg nil  c-state-old-cpp-end nil  c-parse-state-point
33733)

When this output is fed back into Emacs with M-:, it produces an obscure
error message.

This has wasted a lot of time identifying the problem, and fruitlessly
searching for a solution in the Emacs and Elisp manuals, etc.

This is a bug.  If a constant ASCII quote is presented to `message' or
`format', it should be processed as such.  This bug might exist in the
emacs-25 branch, too.  If so, could it be corrected there too, please,
before the release?

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Lars Ingebrigtsen]

Alan Mackenzie <acm@muc.de> writes:

> This has wasted a lot of time identifying the problem, and fruitlessly
> searching for a solution in the Emacs and Elisp manuals, etc.

Wow.  That's pretty egregious:

(message "'foo'")
=> "’foo’"

It's part of Paul grand War On Quotes, I guess.  I think this part
should be reverted, at least.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Drew Adams]

> > This has wasted a lot of time identifying the problem, and fruitlessly
> > searching for a solution in the Emacs and Elisp manuals, etc.
> 
> Wow.  That's pretty egregious: (message "'foo'") => "'foo'"
> 
> It's part of Paul grand War On Quotes, I guess.  I think this part
> should be reverted, at least.

You mean you don't find the improvement more beautiful?  Luddite! ;-)

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 586 bytes --]

(message "%s" M) is the usual way to output a message M without 
formatting it, and something like this is needed anyway if M might 
contain %. Accordingly, I installed the attached patch into the emacs-25 
branch and merged the result to master. I had already adjusted some 
other instances of calls to ‘message’ (to fix problems with % as well as 
with quotes) but missed this call; sorry about that.

The behavior of ‘message’ is documented in the manual, in doc strings, 
and the quoting change in Emacs 25 is documented in etc/NEWS, so it is 
not a bug per se.


[-- Attachment #2: 0001-Fix-quoting-problem-in-cc-engine-debug-message.txt --]
[-- Type: text/plain, Size: 959 bytes --]

From f974345491b418926797559adf853aaedf690e46 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Mon, 2 May 2016 10:06:04 -0700
Subject: [PATCH] Fix quoting problem in cc-engine debug message
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

* lisp/progmodes/cc-engine.el (c-replay-parse-state-state):
Use "%s" format to pass through ‘'’ unscathed (Bug#23425), and
likewise for ‘`’, and ‘%’.
---
 lisp/progmodes/cc-engine.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lisp/progmodes/cc-engine.el b/lisp/progmodes/cc-engine.el
index 2010515..eb015ac 100644
--- a/lisp/progmodes/cc-engine.el
+++ b/lisp/progmodes/cc-engine.el
@@ -3498,7 +3498,7 @@ c-record-parse-state-state
 	   c-state-old-cpp-end
 	   c-parse-state-point))))
 (defun c-replay-parse-state-state ()
-  (message
+  (message "%s"
    (concat "(setq "
     (mapconcat
      (lambda (arg)
-- 
2.5.5

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Alan Mackenzie]

Hello, Paul.

On Mon, May 02, 2016 at 10:53:59AM -0700, Paul Eggert wrote:
> (message "%s" M) is the usual way to output a message M without 
> formatting it, and something like this is needed anyway if M might 
> contain %.

That is the case for a string that might contain percent characters.  It
is not the case for a string known to be without percents.

> Accordingly, I installed the attached patch into the emacs-25 
> branch and merged the result to master.

That is merely tinkering with the symptoms of the problem.  The actual
problem is that `message' corrupts strings.

> I had already adjusted some other instances of calls to ‘message’ (to
> fix problems with % as well as with quotes) but missed this call;
> sorry about that.

There was nothing wrong with `c-replay-parse-state-state'; it used
`message' as documented.  Your change implies that your workaround for
`message''s problems is instead to write something like:

    (message "%s" (format "......." .....))

, which is ridiculous.

> The behavior of ‘message’ is documented in the manual, in doc strings, 

It is not.  There is no mention of its corruption of quotes in
`messages''s doc string.  In the Elisp manual there is just the
following obscure, and somewhat patronising, paragraph:

     In a format string containing single quotes, curved quotes `like
     this' and grave quotes `like this' work better than straight
     quotes 'like this', as `message' typically formats every straight
     quote as a curved closing quote.

Curved quotes do not "work better" than straight quotes, as my
experience this afternoon demonstrates.  Why are readers insulted with
"formats"?  There is no "formatting" going on here; instead some
characters are being expunged, and substituted by other characters.  Why
is the most important idea in that paragraph, this substitution, slid in
as though it was a totally natural thing to do, scarcely worth a
mention?

Why is there no clear definition of what is meant by the confusing terms
"single quotes", "grave quotes", and "straight quotes"?

It appears to me that these substitutions are only done on pairs of
quote marks, not single quotes.  If I'm right, why is this not
documented?

Why is there no mention of `text-quoting-style' on this page?

How do I configure my Emacs so that `message' behaves properly?
Properly, for me, means that it handles percent escape sequences, AND
OTHERWISE LEAVES THE STRING UNCHANGED.  Why is this information not in
the manual and the doc string?

> and the quoting change in Emacs 25 is documented in etc/NEWS, so it is 
> not a bug per se.

Why is it not in the section "incompatible Lisp changes"?

How do I restore the old behaviour of `message'?  This information
surely belongs in etc/NEWS.  As far as I can make out, there is no
setting of `text-quoting-style' equivalent to "do nothing".  Why not?
Not everybody want to "text-quote".

Over the months, I've lost a ridiculously large amount of time to these
curly quotes.  I never want to see them again (even though I'm forced to
in precompiled info files).  Please document how to expunge them.

Can we please fix this before Emacs 25.1 gets released?  Otherwise these
problems will impact LOTS of users.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 1475 bytes --]

On 05/02/2016 12:10 PM, Alan Mackenzie wrote:
>
>> (message "%s" M) is the usual way to output a message M without
>> formatting it, and something like this is needed anyway if M might
>> contain %.
> That is the case for a string that might contain percent characters.  It
> is not the case for a string known to be without percents.

Sure, but the most common problem here is using (message M) to output an 
arbitrary string M that is under user control, which is why the advice 
about (message "%s" M) is helpful. This advice works for all format 
directives, not just % directives.

> There is no mention of its corruption of quotes in
> `messages''s doc string.

Thanks, good point. I fixed it by installing the attached patch, which 
should address the other documentation problems you mention too.

> It appears to me that these substitutions are only done on pairs of
> quote marks, not single quotes.

What gives you that impression? Substitutions are done on all quotes, 
not just paired quotes.

> Properly, for me, means that it handles percent escape sequences, AND
> OTHERWISE LEAVES THE STRING UNCHANGED.

This alternative was considered when designing text-quoting-style, but 
at the time we didn't see a need for the extra complexity. The 
particular glitch you ran into is not a strong argument for revisiting 
this now.

> Why is it not in the section "incompatible Lisp changes"?

It is in that section, under "** New variable 'text-quoting-style' ...".


[-- Attachment #2: 0001-Doc-fixes-for-quoting.txt --]
[-- Type: text/plain, Size: 14304 bytes --]

From ee2f178502594f3c0813b61a64c4da501a8c72fb Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Tue, 3 May 2016 08:02:16 -0700
Subject: [PATCH] Doc fixes for quoting

* doc/emacs/text.texi, doc/lispintro/emacs-lisp-intro.texi:
* doc/lispref/control.texi, doc/lispref/display.texi:
* doc/lispref/help.texi, doc/lispref/strings.texi, lisp/subr.el:
* src/callint.c, src/doprnt.c, src/editfns.c:
Document quoting a bit more systematically.
Problem reported by Alan Mackenzie (Bug#23425).
---
 doc/emacs/text.texi                 |  4 ++--
 doc/lispintro/emacs-lisp-intro.texi | 28 +++++++++++-----------------
 doc/lispref/control.texi            | 10 ++++++----
 doc/lispref/display.texi            | 10 ++++++----
 doc/lispref/help.texi               |  4 ++++
 doc/lispref/strings.texi            | 10 ++++++++--
 lisp/subr.el                        | 14 ++++++++++----
 src/callint.c                       |  2 +-
 src/doprnt.c                        |  6 +++---
 src/editfns.c                       | 13 ++++++++-----
 10 files changed, 59 insertions(+), 42 deletions(-)

diff --git a/doc/emacs/text.texi b/doc/emacs/text.texi
index 66f01b4..579f788 100644
--- a/doc/emacs/text.texi
+++ b/doc/emacs/text.texi
@@ -862,8 +862,8 @@ Text Mode
 
   Text mode turns off the features concerned with comments except when
 you explicitly invoke them.  It changes the syntax table so that
-single-quotes are considered part of words (e.g., @samp{don't} is
-considered one word).  However, if a word starts with a single-quote,
+apostrophes are considered part of words (e.g., @samp{don't} is
+considered one word).  However, if a word starts with an apostrophe,
 it is treated as a prefix for the purposes of capitalization
 (e.g., @kbd{M-c} converts @samp{'hello'} into @samp{'Hello'}, as
 expected).
diff --git a/doc/lispintro/emacs-lisp-intro.texi b/doc/lispintro/emacs-lisp-intro.texi
index 78c1865..eea46af 100644
--- a/doc/lispintro/emacs-lisp-intro.texi
+++ b/doc/lispintro/emacs-lisp-intro.texi
@@ -1004,11 +1004,11 @@ List Processing
 claim is unwarranted.  Lisp stands for LISt Processing, and the
 programming language handles @emph{lists} (and lists of lists) by
 putting them between parentheses.  The parentheses mark the boundaries
-of the list.  Sometimes a list is preceded by a single apostrophe or
-quotation mark, @samp{'}@footnote{The single apostrophe or quotation
-mark is an abbreviation for the function @code{quote}; you need not
-think about functions now; functions are defined in @ref{Making
-Errors, , Generate an Error Message}.}  Lists are the basis of Lisp.
+of the list.  Sometimes a list is preceded by an apostrophe @samp{'},
+called a @dfn{single-quote} in Lisp.@footnote{A single-quote is an
+abbreviation for the special form @code{quote}; you need not think
+about special forms now.  @xref{Complications}.}  Lists are the basis
+of Lisp.
 
 @menu
 * Lisp Lists::                  What are lists?
@@ -2490,14 +2490,7 @@ Summary
 Then the instructions in the function definition are carried out.
 
 @item
-A single quotation mark,
-@ifinfo
-'
-@end ifinfo
-@ifnotinfo
-@code{'}
-@end ifnotinfo
-, tells the Lisp interpreter that it should
+A single-quote @samp{'} tells the Lisp interpreter that it should
 return the following expression as written, and not evaluate it as it
 would if the quote were not there.
 
@@ -7610,7 +7603,8 @@ Complete zap-to-char
 suitable for use as quotes.  On most modern displays this is no longer
 true, and when these two ASCII characters appear in documentation
 strings or diagnostic message formats, Emacs typically transliterates
-them to curved single quotes, so that the abovequoted symbol appears
+them to @dfn{curved quotes} (left and right single quotation marks),
+so that the abovequoted symbol appears
 as @t{‘case-fold-search’}.  Source-code strings can also simply use
 curved quotes directly.
 
@@ -17117,7 +17111,7 @@ Text and Auto-fill
 
 We are already familiar with @code{setq}.  It sets the following variable,
 @code{major-mode}, to the subsequent value, which is @code{text-mode}.
-The single quote mark before @code{text-mode} tells Emacs to deal directly
+The single-quote before @code{text-mode} tells Emacs to deal directly
 with the @code{text-mode} symbol, not with whatever it might stand for.
 @xref{set & setq, , Setting the Value of a Variable},
 for a reminder of how @code{setq} works.
@@ -17284,11 +17278,11 @@ Keybindings
 details.)
 
 The command invoked by the keys is @code{compare-windows}.  Note that
-@code{compare-windows} is preceded by a single quote; otherwise, Emacs
+@code{compare-windows} is preceded by a single-quote; otherwise, Emacs
 would first try to evaluate the symbol to determine its value.
 
 These three things, the double quotation marks, the backslash before
-the @samp{C}, and the single quote mark are necessary parts of
+the @samp{C}, and the single-quote are necessary parts of
 keybinding that I tend to forget.  Fortunately, I have come to
 remember that I should look at my existing @file{.emacs} file, and
 adapt what is there.
diff --git a/doc/lispref/control.texi b/doc/lispref/control.texi
index 3f48c45..75d8d28 100644
--- a/doc/lispref/control.texi
+++ b/doc/lispref/control.texi
@@ -1100,10 +1100,12 @@ Signaling Errors
 error symbol @code{error}, and a list containing the string returned by
 @code{format-message}.
 
-In a format string containing single quotes, curved quotes @t{‘like
-this’} and grave quotes @t{`like this'} work better than straight
-quotes @t{'like this'}, as @code{error} typically formats every
-straight quote as a curved closing quote.
+A format that quotes with grave accents and apostrophes @t{`like
+this'} typically generates curved quotes @t{‘like this’}.  In
+contrast, a format that quotes with only apostrophes @t{'like this'}
+typically generates two closing curved quotes @t{’like this’}, an
+unusual style in English.  @xref{Keys in Documentation}, for how the
+@code{text-quoting-style} variable affects generated quotes.
 
 @strong{Warning:} If you want to use your own string as an error message
 verbatim, don't just write @code{(error @var{string})}.  If @var{string}
diff --git a/doc/lispref/display.texi b/doc/lispref/display.texi
index c80f78c..b0cd873 100644
--- a/doc/lispref/display.texi
+++ b/doc/lispref/display.texi
@@ -265,10 +265,12 @@ Displaying Messages
 The string is also added to the @file{*Messages*} buffer, but without
 text properties (@pxref{Logging Messages}).
 
-In a format string containing single quotes, curved quotes @t{‘like
-this’} and grave quotes @t{`like this'} work better than straight
-quotes @t{'like this'}, as @code{message} typically formats every
-straight quote as a curved closing quote.
+A format that quotes with grave accents and apostrophes @t{`like
+this'} typically generates curved quotes @t{‘like this’}.  In
+contrast, a format that quotes with only apostrophes @t{'like this'}
+typically generates two closing curved quotes @t{’like this’}, an
+unusual style in English.  @xref{Keys in Documentation}, for how the
+@code{text-quoting-style} variable affects generated quotes.
 
 In batch mode, the message is printed to the standard error stream,
 followed by a newline.
diff --git a/doc/lispref/help.texi b/doc/lispref/help.texi
index 58a11f2..b945e43 100644
--- a/doc/lispref/help.texi
+++ b/doc/lispref/help.texi
@@ -335,10 +335,14 @@ Keys in Documentation
 @item ‘
 @itemx `
 (left single quotation mark and grave accent) both stand for a left quote.
+This generates a left single quotation mark, an apostrophe, or a grave
+accent depending on the value of @code{text-quoting-style}.
 
 @item ’
 @itemx '
 (right single quotation mark and apostrophe) both stand for a right quote.
+This generates a right single quotation mark or an apostrophe
+depending on the value of @code{text-quoting-style}.
 
 @item \=
 quotes the following character and is discarded; thus, @samp{\=`} puts
diff --git a/doc/lispref/strings.texi b/doc/lispref/strings.texi
index 9d6613c..c6563f7 100644
--- a/doc/lispref/strings.texi
+++ b/doc/lispref/strings.texi
@@ -834,8 +834,14 @@ Formatting Strings
 This function acts like @code{format}, except it also converts any
 curved single quotes in @var{string} as per the value of
 @code{text-quoting-style}, and treats grave accent (@t{`}) and
-apostrophe (@t{'}) as if they were curved single quotes.  @xref{Keys
-in Documentation}.
+apostrophe (@t{'}) as if they were curved single quotes.
+
+A format that quotes with grave accents and apostrophes @t{`like
+this'} typically generates curved quotes @t{‘like this’}.  In
+contrast, a format that quotes with only apostrophes @t{'like this'}
+typically generates two closing curved quotes @t{’like this’}, an
+unusual style in English.  @xref{Keys in Documentation}, for how the
+@code{text-quoting-style} variable affects generated quotes.
 @end defun
 
 @cindex @samp{%} in format
diff --git a/lisp/subr.el b/lisp/subr.el
index 0f02170..3ac61f9 100644
--- a/lisp/subr.el
+++ b/lisp/subr.el
@@ -290,21 +290,27 @@ ignore
 
 ;; Signal a compile-error if the first arg is missing.
 (defun error (&rest args)
-  "Signal an error, making error message by passing all args to `format'.
+  "Signal an error, making a message by passing args to `format-message'.
 In Emacs, the convention is that error messages start with a capital
 letter but *do not* end with a period.  Please follow this convention
-for the sake of consistency."
+for the sake of consistency.
+
+Note: (error \"%s\" VALUE) makes the message VALUE without
+interpreting format characters like `%', `\\=`', and `\\=''."
   (declare (advertised-calling-convention (string &rest args) "23.1"))
   (signal 'error (list (apply #'format-message args))))
 
 (defun user-error (format &rest args)
-  "Signal a pilot error, making error message by passing all args to `format'.
+  "Signal a pilot error, making a message by passing args to `format-message'.
 In Emacs, the convention is that error messages start with a capital
 letter but *do not* end with a period.  Please follow this convention
 for the sake of consistency.
 This is just like `error' except that `user-error's are expected to be the
 result of an incorrect manipulation on the part of the user, rather than the
-result of an actual problem."
+result of an actual problem.
+
+Note: (user-error \"%s\" VALUE) makes the message VALUE without
+interpreting format characters like `%', `\\=`', and `\\=''."
   (signal 'user-error (list (apply #'format-message format args))))
 
 (defun define-error (name message &optional parent)
diff --git a/src/callint.c b/src/callint.c
index e56f7cd..053ee6c 100644
--- a/src/callint.c
+++ b/src/callint.c
@@ -272,7 +272,7 @@ invoke it.  If KEYS is omitted or nil, the return value of
 {
   /* `args' will contain the array of arguments to pass to the function.
      `visargs' will contain the same list but in a nicer form, so that if we
-     pass it to `Fformat' it will be understandable to a human.  */
+     pass it to `Fformat_message' it will be understandable to a human.  */
   Lisp_Object *args, *visargs;
   Lisp_Object specs;
   Lisp_Object filter_specs;
diff --git a/src/doprnt.c b/src/doprnt.c
index 506bbc3..9d8b783 100644
--- a/src/doprnt.c
+++ b/src/doprnt.c
@@ -46,15 +46,15 @@ along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.  */
    ignored %s and %c conversions.  (See below for the detailed documentation of
    what is supported.)  However, this is okay, as this function is supposed to
    be called from `error' and similar functions, and thus does not need to
-   support features beyond those in `Fformat', which is used by `error' on the
-   Lisp level.  */
+   support features beyond those in `Fformat_message', which is used
+   by `error' on the Lisp level.  */
 
 /* In the FORMAT argument this function supports ` and ' as directives
    that output left and right quotes as per ‘text-quoting style’.  It
    also supports the following %-sequences:
 
    %s means print a string argument.
-   %S is silently treated as %s, for loose compatibility with `Fformat'.
+   %S is treated as %s, for loose compatibility with `Fformat_message'.
    %d means print a `signed int' argument in decimal.
    %o means print an `unsigned int' argument in octal.
    %x means print an `unsigned int' argument in hex.
diff --git a/src/editfns.c b/src/editfns.c
index f0ce4e7..a858e6e 100644
--- a/src/editfns.c
+++ b/src/editfns.c
@@ -3665,10 +3665,11 @@ In batch mode, the message is printed to the standard error stream,
 followed by a newline.
 
 The first argument is a format control string, and the rest are data
-to be formatted under control of the string.  See `format' for details.
+to be formatted under control of the string.  See `format-message' for
+details.
 
-Note: Use (message "%s" VALUE) to print the value of expressions and
-variables to avoid accidentally interpreting `%' as format specifiers.
+Note: (message "%s" VALUE) displays the string VALUE without
+interpreting format characters like `%', `\\=`', and `\\=''.
 
 If the first argument is nil or the empty string, the function clears
 any existing message; this lets the minibuffer contents show.  See
@@ -3696,7 +3697,8 @@ DEFUN ("message-box", Fmessage_box, Smessage_box, 1, MANY, 0,
        doc: /* Display a message, in a dialog box if possible.
 If a dialog box is not available, use the echo area.
 The first argument is a format control string, and the rest are data
-to be formatted under control of the string.  See `format' for details.
+to be formatted under control of the string.  See `format-message' for
+details.
 
 If the first argument is nil or the empty string, clear any existing
 message; let the minibuffer contents show.
@@ -3727,7 +3729,8 @@ If this command was invoked with the mouse, use a dialog box if
 `use-dialog-box' is non-nil.
 Otherwise, use the echo area.
 The first argument is a format control string, and the rest are data
-to be formatted under control of the string.  See `format' for details.
+to be formatted under control of the string.  See `format-message' for
+details.
 
 If the first argument is nil or the empty string, clear any existing
 message; let the minibuffer contents show.
-- 
2.5.5

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Eli Zaretskii]

> Date: Mon, 2 May 2016 19:10:31 +0000
> From: Alan Mackenzie <acm@muc.de>
> Cc: 23425@debbugs.gnu.org
> 
> Hello, Paul.

Hello, Alan.

First, I understand your frustration, and I'm sorry you lost time over
this issue.  This happens sometimes when new developments go against
our muscle memory.

Having said that...

> On Mon, May 02, 2016 at 10:53:59AM -0700, Paul Eggert wrote:
> > (message "%s" M) is the usual way to output a message M without 
> > formatting it, and something like this is needed anyway if M might 
> > contain %.
> 
> That is the case for a string that might contain percent characters.  It
> is not the case for a string known to be without percents.

It is now.  In general, the first argument of 'message' is not copied
to output verbatim, so the advice to use "%s" is a sound one.
Especially when you are going to display a string that is produced
dynamically, and so you don't have complete control on the result.
It's simply dangerous.  We just got one more use case in support of
that advice.

> There was nothing wrong with `c-replay-parse-state-state'; it used
> `message' as documented.  Your change implies that your workaround for
> `message''s problems is instead to write something like:
> 
>     (message "%s" (format "......." .....))
> 
> , which is ridiculous.

I don't see it as ridiculous.  You'd have similar problems (and use
the same solution) if the string produced by 'format' included %
characters, right?

>      In a format string containing single quotes, curved quotes `like
>      this' and grave quotes `like this' work better than straight
>      quotes 'like this', as `message' typically formats every straight
>      quote as a curved closing quote.
> [...]
> Why is there no clear definition of what is meant by the confusing terms
> "single quotes", "grave quotes", and "straight quotes"?

Not sure what you mean: AFAICT, the definition is right there in the
text you cited.

> Why is there no mention of `text-quoting-style' on this page?

Because we don't want to advertise it too much.  It is supposed to be
a kind of "fire escape", not something users must routinely do.  If
that assumption is wrong, then having this prominently mentioned in
the manual will be the least of our troubles.

> How do I configure my Emacs so that `message' behaves properly?

I urge you not to do that.  Because if you do, problems like this one
will slip unnoticed, instead of being fixed in the code.

> Properly, for me, means that it handles percent escape sequences, AND
> OTHERWISE LEAVES THE STRING UNCHANGED.  Why is this information not in
> the manual and the doc string?

AFAICT, 'grave' is that value.  And it is documented.

> > and the quoting change in Emacs 25 is documented in etc/NEWS, so it is 
> > not a bug per se.
> 
> Why is it not in the section "incompatible Lisp changes"?

Because it isn't.  If this is an incompatible change, then every new
features that changes behavior is also an incompatible change.

> How do I restore the old behaviour of `message'?

By setting text-quoting-style.  But again, please don't.

> Can we please fix this before Emacs 25.1 gets released?

What fix would you like to see?  I'm not sure I understand.

Thanks.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Alan Mackenzie]

Hello, Eli.

On Tue, May 03, 2016 at 06:38:16PM +0300, Eli Zaretskii wrote:
> > Date: Mon, 2 May 2016 19:10:31 +0000
> > From: Alan Mackenzie <acm@muc.de>
> > Cc: 23425@debbugs.gnu.org

> > Hello, Paul.

> Hello, Alan.

[ .... ]

> > On Mon, May 02, 2016 at 10:53:59AM -0700, Paul Eggert wrote:
> > > (message "%s" M) is the usual way to output a message M without 
> > > formatting it, and something like this is needed anyway if M might 
> > > contain %.

> > That is the case for a string that might contain percent characters.  It
> > is not the case for a string known to be without percents.

> It is now.  In general, the first argument of 'message' is not copied
> to output verbatim, so the advice to use "%s" is a sound one.
> Especially when you are going to display a string that is produced
> dynamically, and so you don't have complete control on the result.
> It's simply dangerous.  We just got one more use case in support of
> that advice.

> > There was nothing wrong with `c-replay-parse-state-state'; it used
> > `message' as documented.  Your change implies that your workaround for
> > `message''s problems is instead to write something like:

> >     (message "%s" (format "......." .....))

> > , which is ridiculous.

> I don't see it as ridiculous.  You'd have similar problems (and use
> the same solution) if the string produced by 'format' included %
> characters, right?

I suppose so.  Have you got anything agains me putting that suggestion
into etc/NEWS, in the entry for `message'?

> >      In a format string containing single quotes, curved quotes `like
> >      this' and grave quotes `like this' work better than straight
> >      quotes 'like this', as `message' typically formats every straight
> >      quote as a curved closing quote.
> > [...]
> > Why is there no clear definition of what is meant by the confusing terms
> > "single quotes", "grave quotes", and "straight quotes"?

> Not sure what you mean: AFAICT, the definition is right there in the
> text you cited.

Only in the form of characters I can't distinguish on my terminal, and
couldn't even if they were distinct due to my failing eyesight ;-).  And
not in the form I would need to be able to program with them, such as
their ASCII/UTF-8 codes.

> > Why is there no mention of `text-quoting-style' on this page?

> Because we don't want to advertise it too much.  It is supposed to be
> a kind of "fire escape", not something users must routinely do.  If
> that assumption is wrong, then having this prominently mentioned in
> the manual will be the least of our troubles.

If I need to output source code using `message', I will need to bind
`text-quoting-style' to a sensible value, won't I?  That is an argument
for full documentation of it.  And why is it something that users
shouldn't routinely do?  Surely it should be a configuration variable.
Some people, like me, don't like being lied to by a computer program.

> > How do I configure my Emacs so that `message' behaves properly?

> I urge you not to do that.  Because if you do, problems like this one
> will slip unnoticed, instead of being fixed in the code.

I'd like it to be fixed in the code.  :-)

> > Properly, for me, means that it handles percent escape sequences, AND
> > OTHERWISE LEAVES THE STRING UNCHANGED.  Why is this information not in
> > the manual and the doc string?

> AFAICT, 'grave' is that value.  And it is documented.

No, `grave' doesn't work.  This changes curly quotes to ASCII quotes, i.e.
it distorts the input and loses information.  The value wanted, let's
call it `leave', would suppress all substitution of quotes.

> > > and the quoting change in Emacs 25 is documented in etc/NEWS, so it is 
> > > not a bug per se.

> > Why is it not in the section "incompatible Lisp changes"?

> Because it isn't.  If this is an incompatible change, then every new
> features that changes behavior is also an incompatible change.

No.  This particular change causes previously working lisp code (such as
my function from yesterday) to stop working.

> > How do I restore the old behaviour of `message'?

> By setting text-quoting-style.  But again, please don't.

I want my Emacs to deliver messages to me as they were intended by the
people who wrote them.  I have a visceral disgust of and hate for this
distortion of strings caused by this translation of quotes.  I'm sure I'm
not the only person who feels like this (and indeed, there were
snide/sarcastic remarks on this list yesterday in my support).  I want to
configure my Emacs to work right for me.  I don't understand why this
isn't intended to be configurable.

> > Can we please fix this before Emacs 25.1 gets released?

> What fix would you like to see?  I'm not sure I understand.

I want there to be an additional value `leave' for `text-quoting-style'
which would suppress all substitution of quotes.  Something like this:



diff --git a/lisp/help-fns.el b/lisp/help-fns.el
index 6e8a108..b5a2c84 100644
--- a/lisp/help-fns.el
+++ b/lisp/help-fns.el
@@ -626,7 +626,7 @@ describe-function-1
             ;; Avoid asking the user annoying questions if she decides
             ;; to save the help buffer, when her locale's codeset
             ;; isn't UTF-8.
-            (unless (memq text-quoting-style '(straight grave))
+            (unless (memq text-quoting-style '(leave straight grave))
               (set-buffer-file-coding-system 'utf-8))))))))
 
 ;; Add defaults to `help-fns-describe-function-functions'.
diff --git a/src/doc.c b/src/doc.c
index 7cdb0d0..b858e64 100644
--- a/src/doc.c
+++ b/src/doc.c
@@ -704,6 +704,8 @@ text_quoting_style (void)
       ? default_to_grave_quoting_style ()
       : EQ (Vtext_quoting_style, Qgrave))
     return GRAVE_QUOTING_STYLE;
+  else if (EQ (Vtext_quoting_style, Qleave))
+    return LEAVE_QUOTING_STYLE;
   else if (EQ (Vtext_quoting_style, Qstraight))
     return STRAIGHT_QUOTING_STYLE;
   else
@@ -988,7 +990,8 @@ Otherwise, return a new string.  */)
 	  int ch = STRING_CHAR_AND_LENGTH (strp, len);
 	  if ((ch == LEFT_SINGLE_QUOTATION_MARK
 	       || ch == RIGHT_SINGLE_QUOTATION_MARK)
-	      && quoting_style != CURVE_QUOTING_STYLE)
+	      && quoting_style != CURVE_QUOTING_STYLE
+              && quoting_style != LEAVE_QUOTING_STYLE)
 	    {
 	      *bufp++ = ((ch == LEFT_SINGLE_QUOTATION_MARK
 			  && quoting_style == GRAVE_QUOTING_STYLE)
@@ -1033,6 +1036,7 @@ void
 syms_of_doc (void)
 {
   DEFSYM (Qfunction_documentation, "function-documentation");
+  DEFSYM (Qleave, "leave");
   DEFSYM (Qgrave, "grave");
   DEFSYM (Qstraight, "straight");
 
@@ -1046,7 +1050,11 @@ syms_of_doc (void)
 
   DEFVAR_LISP ("text-quoting-style", Vtext_quoting_style,
                doc: /* Style to use for single quotes in help and messages.
-Its value should be a symbol.
+Its value should be a symbol.  It works by substituting certain single
+quotes for certain other single quotes.  This is done in help output and
+`message' output.  It is not done in `format'.
+
+`leave' means do not do any substitutions.
 `curve' means quote with curved single quotes \\=‘like this\\=’.
 `straight' means quote with straight apostrophes \\='like this\\='.
 `grave' means quote with grave accent and apostrophe \\=`like this\\='.
diff --git a/src/editfns.c b/src/editfns.c
index f0ce4e7..415d3c3 100644
--- a/src/editfns.c
+++ b/src/editfns.c
@@ -3923,6 +3923,8 @@ styled_format (ptrdiff_t nargs, Lisp_Object *args, bool message)
       multibyte = true;
 
   int quoting_style = message ? text_quoting_style () : -1;
+  if (quoting_style == LEAVE_QUOTING_STYLE)
+    quoting_style = -1;
 
   /* If we start out planning a unibyte result,
      then discover it has to be multibyte, we jump back to retry.  */
diff --git a/src/lisp.h b/src/lisp.h
index 6a98adb..6a53671 100644
--- a/src/lisp.h
+++ b/src/lisp.h
@@ -4183,6 +4183,9 @@ extern void syms_of_callproc (void);
 /* Defined in doc.c.  */
 enum text_quoting_style
   {
+    /* Leave quotes unchanged.  */
+    LEAVE_QUOTING_STYLE,
+
     /* Use curved single quotes ‘like this’.  */
     CURVE_QUOTING_STYLE,
 

> Thanks.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Drew Adams]

> we didn't see a need for the extra complexity

Talk about the tail wagging the dog!  This whole quote-beautification
crusade (yes, the term is appropriate) has been a huge boatload of
extra complexity.  And it's not even clear that we've seen the end of it.

Sure, I know the mantra: "That ship has already sailed" ("Shut up and color").

This particular ship is a gigantic barge of sludge, and it seems it's
trying to plod through ice fields.  Global warming to the rescue?

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Eli Zaretskii]

> Date: Tue, 3 May 2016 16:27:16 +0000
> Cc: eggert@cs.ucla.edu, 23425@debbugs.gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> > >     (message "%s" (format "......." .....))
> 
> > > , which is ridiculous.
> 
> > I don't see it as ridiculous.  You'd have similar problems (and use
> > the same solution) if the string produced by 'format' included %
> > characters, right?
> 
> I suppose so.  Have you got anything agains me putting that suggestion
> into etc/NEWS, in the entry for `message'?

No, I have nothing against that.

> > >      In a format string containing single quotes, curved quotes `like
> > >      this' and grave quotes `like this' work better than straight
> > >      quotes 'like this', as `message' typically formats every straight
> > >      quote as a curved closing quote.
> > > [...]
> > > Why is there no clear definition of what is meant by the confusing terms
> > > "single quotes", "grave quotes", and "straight quotes"?
> 
> > Not sure what you mean: AFAICT, the definition is right there in the
> > text you cited.
> 
> Only in the form of characters I can't distinguish on my terminal, and
> couldn't even if they were distinct due to my failing eyesight ;-).  And
> not in the form I would need to be able to program with them, such as
> their ASCII/UTF-8 codes.

Well, a picture is worth about a thousand words.  I very much doubt
one could explain that clearly enough, if we assume the characters are
not displayed correctly and that the user isn't already familiar with
the terminology.  How do you "define" character by just talking about
them, without showing their glyphs??

> > > Why is there no mention of `text-quoting-style' on this page?
> 
> > Because we don't want to advertise it too much.  It is supposed to be
> > a kind of "fire escape", not something users must routinely do.  If
> > that assumption is wrong, then having this prominently mentioned in
> > the manual will be the least of our troubles.
> 
> If I need to output source code using `message', I will need to bind
> `text-quoting-style' to a sensible value, won't I?

No, you just need to use "%s" as the first argument, that's all.

> And why is it something that users shouldn't routinely do?

Because the theory behind this change is that everyone will want that,
certainly after they get used to it a little.  If that theory is
wrong, we will have to change much more than just document that
variable or teach users to use it more.

> Surely it should be a configuration variable.  Some people, like me,
> don't like being lied to by a computer program.

You are not being lied to.  Just think of ` and ' as yet another
format specifier character, not unlike %.

> > > How do I configure my Emacs so that `message' behaves properly?
> 
> > I urge you not to do that.  Because if you do, problems like this one
> > will slip unnoticed, instead of being fixed in the code.
> 
> I'd like it to be fixed in the code.  :-)

You mean, disable the conversion?  Not going to happen before 25.1,
and unless there's a user outcry, not going to happen, period.

That's why I think we should use the default, and find and fix any
leftovers like the one which started this discussion.

> > > Properly, for me, means that it handles percent escape sequences, AND
> > > OTHERWISE LEAVES THE STRING UNCHANGED.  Why is this information not in
> > > the manual and the doc string?
> 
> > AFAICT, 'grave' is that value.  And it is documented.
> 
> No, `grave' doesn't work.  This changes curly quotes to ASCII quotes, i.e.
> it distorts the input and loses information.

??? Are we talking about the same issue?

  emacs -Q
  M-: (setq text-quoting-style 'grave) RET
  M-: (message "`this'") RET => `this'

What am I missing?

> > > Why is it not in the section "incompatible Lisp changes"?
> 
> > Because it isn't.  If this is an incompatible change, then every new
> > features that changes behavior is also an incompatible change.
> 
> No.  This particular change causes previously working lisp code (such as
> my function from yesterday) to stop working.

Like any other change can.

> > > How do I restore the old behaviour of `message'?
> 
> > By setting text-quoting-style.  But again, please don't.
> 
> I want my Emacs to deliver messages to me as they were intended by the
> people who wrote them.  I have a visceral disgust of and hate for this
> distortion of strings caused by this translation of quotes.  I'm sure I'm
> not the only person who feels like this (and indeed, there were
> snide/sarcastic remarks on this list yesterday in my support).

You are not just anyone.  You are a core developer and a maintainer of
several important packages.  That is a responsibility that places an
onus on us to "suffer" a little so that bugs will be detected and
fixed sooner rather than latter.  In general, developers should try
using as many defaults as they can, for that very reason -- to see the
problems that users will as soon as possible.

If you still don't want that, I cannot force you.  It's just that
there's so few of us that every single one counts.

> > > Can we please fix this before Emacs 25.1 gets released?
> 
> > What fix would you like to see?  I'm not sure I understand.
> 
> I want there to be an additional value `leave' for `text-quoting-style'
> which would suppress all substitution of quotes.

See above: I'm missing something here, because I don't understand why
is that needed.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Eli Zaretskii]

> Date: Tue, 3 May 2016 09:36:10 -0700 (PDT)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: 23425@debbugs.gnu.org
> 
> > we didn't see a need for the extra complexity
> 
> Talk about the tail wagging the dog!  This whole quote-beautification
> crusade (yes, the term is appropriate) has been a huge boatload of
> extra complexity.  And it's not even clear that we've seen the end of it.
> 
> Sure, I know the mantra: "That ship has already sailed" ("Shut up and color").
> 
> This particular ship is a gigantic barge of sludge, and it seems it's
> trying to plod through ice fields.  Global warming to the rescue?

Please drop the attitude.  This isn't helping a bit.  It does waste
precious time and bandwidth.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Alan Mackenzie]

Hello, Eli.

On Tue, May 03, 2016 at 07:55:45PM +0300, Eli Zaretskii wrote:
> > Date: Tue, 3 May 2016 16:27:16 +0000
> > Cc: eggert@cs.ucla.edu, 23425@debbugs.gnu.org
> > From: Alan Mackenzie <acm@muc.de>

> > I suppose so.  Have you got anything agains me putting that
> > suggestion [ the trick of using `format' inside `message] into
> > etc/NEWS, in the entry for `message'?

> No, I have nothing against that.

OK, I'll do that later.

> > > Not sure what you mean: AFAICT, the definition is right there in the
> > > text you cited.

> > Only in the form of characters I can't distinguish on my terminal, and
> > couldn't even if they were distinct due to my failing eyesight ;-).  And
> > not in the form I would need to be able to program with them, such as
> > their ASCII/UTF-8 codes.

> Well, a picture is worth about a thousand words.  I very much doubt
> one could explain that clearly enough, if we assume the characters are
> not displayed correctly and that the user isn't already familiar with
> the terminology.  How do you "define" character by just talking about
> them, without showing their glyphs??

This isn't simple.  I think what's needed is a clear explanation of the
conversion process, not merely the result of using it - I'm still a
little confused over whether a left ASCII quote (`) might get converted
into a right ASCII or curly quote.  (A little later - yes it can happen)
I don't think that is clearly documented anywhere.

> > > > Why is there no mention of `text-quoting-style' on this page?

> > > Because we don't want to advertise it too much.  It is supposed to be
> > > a kind of "fire escape", not something users must routinely do.  If
> > > that assumption is wrong, then having this prominently mentioned in
> > > the manual will be the least of our troubles.

> > And why is it something that users shouldn't routinely do?

> Because the theory behind this change is that everyone will want that,
> certainly after they get used to it a little.

That seems a very strange attitude from an Emacs developer - that one
developer's personal preferences should be imposed on all Emacs users
without the possibility of disabling by configuration.

It is quite clear from posts in emacs-devel and bug-gnu-emacs over the
last year or so that NOT EVERYBODY wants these curly quote changes.  Far
from it.  Just like not everybody wants light text on a dark background,
or (thinking back to Emacs 21) not everybody wants fringes on their GUI
screen.  Users were complaining for years about the unconfigurable
fringes, until Emacs 22 was released.

> If that theory is wrong, we will have to change much more than just
> document that variable or teach users to use it more.

That theory _is_ wrong, certainly amongst Emacs developers.  Even in this
bug thread, two other people have expressed negative sentiments about
this matter.

> > Surely it should be a configuration variable.  Some people, like me,
> > don't like being lied to by a computer program.

> You are not being lied to.  Just think of ` and ' as yet another
> format specifier character, not unlike %.

They're not - they're literal characters.  The quote which should have
been output yesterday afternoon, but got corrupted, was certainly a
literal quote.

> > > I urge you not to do that.  Because if you do, problems like this one
> > > will slip unnoticed, instead of being fixed in the code.

> > I'd like it to be fixed in the code.  :-)

> You mean, disable the conversion?  Not going to happen before 25.1,
> and unless there's a user outcry, not going to happen, period.

No, I mean make this conversion optional and configurable, as in the
patch in my last post.

> That's why I think we should use the default, and find and fix any
> leftovers like the one which started this discussion.

This particular default just makes me too angry to get work done.  I'm
still using 24.5.  After becoming tired out in the fight over curly
quotes last summer/autumn, I just left that task to other people.  I
assumed that the matter of curly quotes had been sorted out
satisfactorally.

[ .... ]

> > No, `grave' doesn't work.  This changes curly quotes to ASCII quotes, i.e.
> > it distorts the input and loses information.

> ??? Are we talking about the same issue?

>   emacs -Q
>   M-: (setq text-quoting-style 'grave) RET
>   M-: (message "`this'") RET => `this'

> What am I missing?

That in that scenario, (message "`this'") ; with curly quotes
gets converted to "`this'" ; with ASCII quotes.  This is not wanted.

[ .... ]

> > > > Can we please fix this before Emacs 25.1 gets released?

> > > What fix would you like to see?  I'm not sure I understand.

> > I want there to be an additional value `leave' for `text-quoting-style'
> > which would suppress all substitution of quotes.

> See above: I'm missing something here, because I don't understand why
> is that needed.

Backwards compatibility, perhaps?

Another reason is the difficulty of discovering from the available
documentation, which is not concise and coherent[*], precisely what the
quote substitution does.  At least on disabling the substitution, you
know where you stand.

[*] There is nowhere any text like "when `message' runs, it subtitutes
ASCII and curly quotes as follow: ....".

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 2476 bytes --]

On 05/03/2016 01:03 PM, Alan Mackenzie wrote:
> On Tue, May 03, 2016 at 07:55:45PM +0300, Eli Zaretskii wrote:
>> No, I have nothing against that.
> OK, I'll do that later.

I gave this a shot by installing the attached; please feel free to 
improve it.

> I'm still a
> little confused over whether a left ASCII quote (`) might get converted
> into a right ASCII or curly quote.

Grave accent (`) might be left alone or converted to apostrophe (') or 
converted to left single quotation mark (‘), depending on whether 
text-quoting-style is ‘grave’, ‘straight’, or ‘curve’, respectively. 
Grave accent is never converted to right single quotation mark (’).

 > I mean make this conversion optional and configurable, as in the
 > patch in my last post.

This conversion already configurable, in the sense that one can use 
either (setq text-quoting-style 'grave) or (setq text-quoting-style 
'straight) to get behavior that would have worked for the use case that 
prompted this bug report. I haven't seen a realistic use case to 
motivate the proposed extra complexity of another configuration option here.

> I don't think that is clearly documented anywhere.

The current version of the documentation says the following. If this is 
not clear enough, we should improve it so that it is clear.

@item ‘
@itemx `
(left single quotation mark and grave accent) both stand for a left quote.
This generates a left single quotation mark, an apostrophe, or a grave
accent depending on the value of @code{text-quoting-style}.

@item ’
@itemx '
(right single quotation mark and apostrophe) both stand for a right quote.
This generates a right single quotation mark or an apostrophe
depending on the value of @code{text-quoting-style}.

...

@defvar text-quoting-style
@cindex curved quotes
@cindex curly quotes
The value of this variable is a symbol that specifies the style Emacs
should use for single quotes in the wording of help and messages.
If the variable's value is @code{curve}, the style is
@t{‘like this’} with curved single quotes.  If the value is
@code{straight}, the style is @t{'like this'} with straight
apostrophes.  If the value is @code{grave}, the style is @t{`like
this'} with grave accent and apostrophe, the standard style
before Emacs version 25.  The default value @code{nil}
acts like @code{curve} if curved single quotes are displayable, and
like @code{grave} otherwise.


[-- Attachment #2: 0001-etc-NEWS-Mention-message-s-format.patch --]
[-- Type: application/x-patch, Size: 918 bytes --]

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Eli Zaretskii]

> Date: Tue, 3 May 2016 20:03:30 +0000
> Cc: eggert@cs.ucla.edu, 23425@debbugs.gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> > > And why is it something that users shouldn't routinely do?
> 
> > Because the theory behind this change is that everyone will want that,
> > certainly after they get used to it a little.
> 
> That seems a very strange attitude from an Emacs developer - that one
> developer's personal preferences should be imposed on all Emacs users
> without the possibility of disabling by configuration.
> 
> It is quite clear from posts in emacs-devel and bug-gnu-emacs over the
> last year or so that NOT EVERYBODY wants these curly quote changes.  Far
> from it.  Just like not everybody wants light text on a dark background,
> or (thinking back to Emacs 21) not everybody wants fringes on their GUI
> screen.  Users were complaining for years about the unconfigurable
> fringes, until Emacs 22 was released.

This is unfair on so many levels, that I simply lost any motivation to
continue this discussion.

> > If that theory is wrong, we will have to change much more than just
> > document that variable or teach users to use it more.
> 
> That theory _is_ wrong, certainly amongst Emacs developers.  Even in this
> bug thread, two other people have expressed negative sentiments about
> this matter.

So we have 3 people against that.  Is it really enough?

> > > Surely it should be a configuration variable.  Some people, like me,
> > > don't like being lied to by a computer program.
> 
> > You are not being lied to.  Just think of ` and ' as yet another
> > format specifier character, not unlike %.
> 
> They're not - they're literal characters.

So is %.

> >   emacs -Q
> >   M-: (setq text-quoting-style 'grave) RET
> >   M-: (message "`this'") RET => `this'
> 
> > What am I missing?
> 
> That in that scenario, (message "`this'") ; with curly quotes
> gets converted to "`this'" ; with ASCII quotes.  This is not wanted.

Not sure why.  But anyway, the change such as what you suggest is too
pervasive to install on the emacs-25 branch, unless we want to delay
the release.  Too bad these issues didn't come up in time.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Alan Mackenzie]

Hello, Eli.

On Wed, May 04, 2016 at 06:32:40PM +0300, Eli Zaretskii wrote:
> > Date: Tue, 3 May 2016 20:03:30 +0000
> > Cc: eggert@cs.ucla.edu, 23425@debbugs.gnu.org
> > From: Alan Mackenzie <acm@muc.de>

> > > > And why is it something that users shouldn't routinely do?

> > > Because the theory behind this change is that everyone will want that,
> > > certainly after they get used to it a little.

[ .... ]

> > > If that theory is wrong, we will have to change much more than just
> > > document that variable or teach users to use it more.

> > That theory _is_ wrong, certainly amongst Emacs developers.  Even in this
> > bug thread, two other people have expressed negative sentiments about
> > this matter.

> So we have 3 people against that.  Is it really enough?

That's three people in this thread.  Over the months, more people have
expressed unhappinness about curly quotes, in the various ways they have
been imposed on us.  Yes, I think that is enough.  The comments from the
other two people have been less than fully respectful, and at least one
of them is not known for making snide remarks.  What does that tell you?

Put frankly, the curly quote stuff was installed into Emacs without there
being any consensus in favour of it, and was deliberately done in such a
way that it could not be disabled (see below).

> > > > Surely it should be a configuration variable.  Some people, like me,
> > > > don't like being lied to by a computer program.

> > > You are not being lied to.  Just think of ` and ' as yet another
> > > format specifier character, not unlike %.

> > They're not - they're literal characters.

> So is %.

You can escape a % by writing %%.  That will consistently be output as %.
There is no comparable form of escape for the quote characters.

> > >   emacs -Q
> > >   M-: (setq text-quoting-style 'grave) RET
> > >   M-: (message "`this'") RET => `this'

> > > What am I missing?

> > That in that scenario, (message "`this'") ; with curly quotes
> > gets converted to "`this'" ; with ASCII quotes.  This is not wanted.

> Not sure why.  But anyway, the change such as what you suggest is too
> pervasive to install on the emacs-25 branch, unless we want to delay
> the release.  Too bad these issues didn't come up in time.

I brought this matter up in June 2015, immediately after mandatory
mangling of quotes was introduced.  I installed a patch
(52c3946c872c8bd96508f74cdda5cbb90c664306, on 2015-06-18) to make this
mangling optional.  The very next day, you-know-who effectively reverted
this change, under the guise of expanding the choice available, making
the mangling mandatory again.  I should have fought the issue at the
time, but didn't.

If it is too late to go into 25.1, can we put a fix into 25.2?  And yet
again, I suggest `text-quoting-style' should become a configurable
option, giving users the ability to chose how THEY want messages to be
displayed.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Eli Zaretskii]

> Date: Wed, 4 May 2016 16:48:49 +0000
> Cc: eggert@cs.ucla.edu, 23425@debbugs.gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> > > That theory _is_ wrong, certainly amongst Emacs developers.  Even in this
> > > bug thread, two other people have expressed negative sentiments about
> > > this matter.
> 
> > So we have 3 people against that.  Is it really enough?
> 
> That's three people in this thread.  Over the months, more people have
> expressed unhappinness about curly quotes, in the various ways they have
> been imposed on us.  Yes, I think that is enough.

I don't.  Please let's wait until 25.1 is released, the only way to
change this decision is if we see a lot of complaints about this.

> The comments from the other two people have been less than fully
> respectful, and at least one of them is not known for making snide
> remarks.  What does that tell you?

That some people don't like this.  But we knew that back when it was
decided to have this feature.

> Put frankly, the curly quote stuff was installed into Emacs without there
> being any consensus in favour of it, and was deliberately done in such a
> way that it could not be disabled (see below).

Not true.  If there were no consensus, at least not a wide enough one,
this wouldn't have been admitted.  The objections were not numerous
enough and strong enough to prevent these changes.

> > > They're not - they're literal characters.
> 
> > So is %.
> 
> You can escape a % by writing %%.  That will consistently be output as %.
> There is no comparable form of escape for the quote characters.

Yes, there is: use "%s" as the first argument to 'message'.  Which is
a Good Thing anyway, as we all know.

> If it is too late to go into 25.1, can we put a fix into 25.2?

I see no reason not to install a change along these lines on master,
if the new value is not the default.

> And yet again, I suggest `text-quoting-style' should become a
> configurable option, giving users the ability to chose how THEY want
> messages to be displayed.

And I yet again urge you to wait until 25.1 hits the streets, and we
see what the Emacs community at large thinks about this.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Josh]

On Wed, May 4, 2016 at 10:03 AM, Eli Zaretskii wrote:
>> > > That theory _is_ wrong, certainly amongst Emacs developers.  Even in this
>> > > bug thread, two other people have expressed negative sentiments about
>> > > this matter.
>>
>> > So we have 3 people against that.  Is it really enough?
>>
>> That's three people in this thread.  Over the months, more people have
>> expressed unhappinness about curly quotes, in the various ways they have
>> been imposed on us.  Yes, I think that is enough.
>
> I don't.  Please let's wait until 25.1 is released, the only way to
> change this decision is if we see a lot of complaints about this.

And how do you expect to receive that feedback?  Through bug reports?
Can I file mine now instead of waiting for the release of 25.1?

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Alan Mackenzie]

Hello, Eli.

On Wed, May 04, 2016 at 08:03:43PM +0300, Eli Zaretskii wrote:
> > Date: Wed, 4 May 2016 16:48:49 +0000
> > Cc: eggert@cs.ucla.edu, 23425@debbugs.gnu.org
> > From: Alan Mackenzie <acm@muc.de>

> > > > That theory _is_ wrong, certainly amongst Emacs developers.  Even in this
> > > > bug thread, two other people have expressed negative sentiments about
> > > > this matter.

> > > So we have 3 people against that.  Is it really enough?

> > That's three people in this thread.  Over the months, more people have
> > expressed unhappinness about curly quotes, in the various ways they have
> > been imposed on us.  Yes, I think that is enough.

> I don't.  Please let's wait until 25.1 is released, the only way to
> change this decision is if we see a lot of complaints about this.

There won't be lots of complaints about this.  There will be a few people
who notice, and some of them will utterly hate it, just as I do.  Some
will have their Lisp rendered unworkable, just as happened to me.  There
are thousands of options in Emacs, and most users care about only a small
subset (although the subset is different for each user).  You could
remove any option, no matter how "important", and you'd get only a small
number of complaints.  This doesn't seem to me like the way to judge
things.

> > The comments from the other two people have been less than fully
> > respectful, and at least one of them is not known for making snide
> > remarks.  What does that tell you?

> That some people don't like this.  But we knew that back when it was
> decided to have this feature.

And when was this decision taken?  The answer is it wasn't.  Despite the
great opposition to it, Paul plowed ahead with several varied schemes for
curly quotes, not waiting for consensus first, and in the end, people
just got tired of opposing him.

> > Put frankly, the curly quote stuff was installed into Emacs without there
> > being any consensus in favour of it, and was deliberately done in such a
> > way that it could not be disabled (see below).

> Not true.  If there were no consensus, at least not a wide enough one,
> this wouldn't have been admitted.

That's not the way the Emacs project works.  Most contributors work on a
cooperative basis and first discuss contentious things and abide by
understandings reached.  You and I certainly do.  That leaves us open to
abuse by aggressive committers, who don't abide by the above convention.
This is what happened with curly quotes.  There is nobody in the project
who feels empowered to revert "premature" commits.

> The objections were not numerous enough and strong enough to prevent
> these changes.

The objections to curly quotes were extremely numerous (I've spent some
time today looking at old threads), and support for them was sparse.

[ .... ]

> > If it is too late to go into 25.1, can we put a fix into 25.2?

> I see no reason not to install a change along these lines on master,
> if the new value is not the default.

We'll see what we can manage.

> > And yet again, I suggest `text-quoting-style' should become a
> > configurable option, giving users the ability to chose how THEY want
> > messages to be displayed.

> And I yet again urge you to wait until 25.1 hits the streets, and we
> see what the Emacs community at large thinks about this.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Paul Eggert]

On 05/04/2016 11:04 AM, Alan Mackenzie wrote:
> Despite the
> great opposition to it, Paul plowed ahead with several varied schemes for
> curly quotes, not waiting for consensus first,

This misrepresents what happened. I proposed a scheme, got feedback, 
altered the proposal in response to the feedback, gained general (though 
not universal) agreement, and got an OK from the maintainer before 
committing the change. The installed scheme differs quite a bit from my 
personal preference. Like most compromises it's easy for each of us to 
find something we dislike about it.

I plan to look into the further change you proposed, with an eye toward 
installing something like it (but I hope simpler) in the master branch. 
It's a bit late to be fiddling with this sort of thing in the emacs-25 
branch, though.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[John Wiegley]

[-- Attachment #1: Type: text/plain, Size: 3465 bytes --]

>>>>> Alan Mackenzie <acm@muc.de> writes:

> There won't be lots of complaints about this. There will be a few people who
> notice, and some of them will utterly hate it, just as I do. Some will have
> their Lisp rendered unworkable, just as happened to me. There are thousands
> of options in Emacs, and most users care about only a small subset (although
> the subset is different for each user). You could remove any option, no
> matter how "important", and you'd get only a small number of complaints.
> This doesn't seem to me like the way to judge things.

Hi guys,

I appreciate the passion on this issue; I had a feeling this would come up,
and I think the reaction is going to be far more widespread than some expect.

As maintainer, I'll go on record by saying that I very much dislike the new
curly quotes logic. After the reactions I've read here, I'm convinced again
that a customization option needs to exist in 25.1, so we can easily recommend
it to people who react as Alan has done.

I realize it's harder to take away a defcustom than to promote a defvar, but
this is too much of a hot issue to wait an entire release for sufficient bug
reports to come in before we act. If we think this reaction is going to be
isolated to just a few developers here, I fear we've underestimated the global
capacity for bike-shedding: and nothing says bike-shedding better than a
purely aesthetic concern as minor as ' vs. ’.

In fact, I would like to rip all the curly-quote redisplay logic out of Emacs,
for several reasons:

  1. It achieves nothing technically. Emacs is not a better editor because of
     it in any objective way; it is purely an aesthetic change, desired only
     by those who agree with its aesthetics.

  2. It has caused numerous bugs already.

  3. It has provoked heated debate many times already. The amount of time
     lost on emacs-devel to this one issue is an absolute shame.

  4. It changes Emacs according to the preferences of specific people, without
     giving others -- in the very spirit of Emacs! -- an option to choose
     one's desired behavior in the standard fashion.

At this point, I cannot undo this feature, since that would destabilize 25.1.
However, we must have an option to conveniently disable it before the release
happens. Of that I am now certain. Would someone be willing to craft a patch,
please?


Lastly, to play devil's advocate: I also recognize, after having written this,
that I'm something of an "old fogey" in Emacs terms, with a special place for
ASCII in my heart. It's quite possible that the new-and-coming generation is
very fond of Unicode, and they will all love this feature, since it gives
Emacs a more modern "feel". For their sake, I'm prepared to wait for the
world's judgment before deciding that this feature is entirely terrible.

But for the sake of my fellow dinosaurs, I need a way to turn it off! Please!
They are like tiny knives, stabbing at my mind with their Unicode edges; or a
plague of typeset locusts, foretelling the soon-to-come Apostroclypse.


I hope we can resolve this discussion with less ire, and more focus on what
will make 25.1 a better release. No one is ill-intended here. We just have
very different opinions on what looks nice.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 629 bytes --]

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Alan Mackenzie]

Hello, Paul.

On Wed, May 04, 2016 at 11:39:08AM -0700, Paul Eggert wrote:
> On 05/04/2016 11:04 AM, Alan Mackenzie wrote:
> > Despite the
> > great opposition to it, Paul plowed ahead with several varied schemes for
> > curly quotes, not waiting for consensus first,

> This misrepresents what happened.

You installed your contentious changes without prior discussion on
emacs-devel.

> I proposed a scheme, ....

By which you mean you installed your changes on the master branch.

> ... got feedback, altered the proposal in response to the feedback,
> gained general (though not universal) agreement, ....

It's a pity you didn't attempt to garner that agreement BEFORE
installing your changes.  That might have saved a lot of bad feeling.

> and got an OK from the maintainer before committing the change. The
> installed scheme differs quite a bit from my personal preference. Like
> most compromises it's easy for each of us to find something we dislike
> about it.

You'll recall that at one point I installed a quick change to make the
translation of quotes optional.  Your reaction, the very next day, was
to nullify my change, rendering your translations mandatory again.

> I plan to look into the further change you proposed, with an eye toward 
> installing something like it (but I hope simpler) in the master branch. 
> It's a bit late to be fiddling with this sort of thing in the emacs-25 
> branch, though.

No, that won't do.  By "something like it", you probably mean something
that leaves the translation mandatory.  Only a facility which allows
total disablement will be adequate here.  After discussion with Eli, I
intend to apply the patch I proposed yesterday, and I'd be much obliged
if this time you would allow its intention to survive.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[John Wiegley]

[-- Attachment #1: Type: text/plain, Size: 633 bytes --]

>>>>> Alan Mackenzie <acm@muc.de> writes:

>> This misrepresents what happened.
> You installed your contentious changes without prior discussion on
> emacs-devel.

Please, do not let this bug devolve into an argument between volunteers.  I
cannot conceive of a world in which Paul did any of this maliciously, nor am I
insensitive to your concerns, Alan.

Let's focus on what we're going to do about it now, not what we might have
done about it back then.

Thanks,
-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 629 bytes --]

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Eli Zaretskii]

> Date: Wed, 4 May 2016 10:37:33 -0700
> From: Josh <josh+gnu@nispio.net>
> Cc: Alan Mackenzie <acm@muc.de>, eggert@cs.ucla.edu, 23425@debbugs.gnu.org
> 
> > I don't.  Please let's wait until 25.1 is released, the only way to
> > change this decision is if we see a lot of complaints about this.
> 
> And how do you expect to receive that feedback?  Through bug reports?

Yes.  Also, through messages on help-gnu-emacs and here.

> Can I file mine now instead of waiting for the release of 25.1?

Yes, of course.  But didn't you do that already?  This is the bug
report about that very feature.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Eli Zaretskii]

> Date: Wed, 4 May 2016 18:04:17 +0000
> Cc: eggert@cs.ucla.edu, 23425@debbugs.gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> > I don't.  Please let's wait until 25.1 is released, the only way to
> > change this decision is if we see a lot of complaints about this.
> 
> There won't be lots of complaints about this.  There will be a few people
> who notice, and some of them will utterly hate it, just as I do.  Some
> will have their Lisp rendered unworkable, just as happened to me.  There
> are thousands of options in Emacs, and most users care about only a small
> subset (although the subset is different for each user).  You could
> remove any option, no matter how "important", and you'd get only a small
> number of complaints.  This doesn't seem to me like the way to judge
> things.

I see no other way.

> > That some people don't like this.  But we knew that back when it was
> > decided to have this feature.
> 
> And when was this decision taken?  The answer is it wasn't.  Despite the
> great opposition to it, Paul plowed ahead with several varied schemes for
> curly quotes, not waiting for consensus first, and in the end, people
> just got tired of opposing him.
> 
> > > Put frankly, the curly quote stuff was installed into Emacs without there
> > > being any consensus in favour of it, and was deliberately done in such a
> > > way that it could not be disabled (see below).
> 
> > Not true.  If there were no consensus, at least not a wide enough one,
> > this wouldn't have been admitted.
> 
> That's not the way the Emacs project works.  Most contributors work on a
> cooperative basis and first discuss contentious things and abide by
> understandings reached.  You and I certainly do.  That leaves us open to
> abuse by aggressive committers, who don't abide by the above convention.
> This is what happened with curly quotes.  There is nobody in the project
> who feels empowered to revert "premature" commits.

That's not my recollection of how this happened.  But the records are
all there, so anyone can re-read them and make up their own minds.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Dmitry Gutov]

On 05/04/2016 09:46 PM, John Wiegley wrote:

> As maintainer, I'll go on record by saying that I very much dislike the new
> curly quotes logic. After the reactions I've read here, I'm convinced again
> that a customization option needs to exist in 25.1, so we can easily recommend
> it to people who react as Alan has done.

This doesn't seem like a good idea to me.

If we add a custom option, doesn't that constitute a promise that the 
curly quote translation *will* work for at least several releases of 
Emacs after that?

Because the option would have a value meaning "translate straight quotes 
to curlies", and we'll have to support it once the option is introduced.

> In fact, I would like to rip all the curly-quote redisplay logic out of Emacs,
> for several reasons:

I would too (though I wouldn't actually mind it if it were implemented 
in a very different fashion). But this is a totally different direction 
from "let's add a user option". I think.

>   3. It has provoked heated debate many times already. The amount of time
>      lost on emacs-devel to this one issue is an absolute shame.

Word.

>   4. It changes Emacs according to the preferences of specific people, without
>      giving others -- in the very spirit of Emacs! -- an option to choose
>      one's desired behavior in the standard fashion.
>
> At this point, I cannot undo this feature, since that would destabilize 25.1.

Can we actually leave it in, while all but promising to take it out in 
the next release?

That would seem like an odd choice.

> Lastly, to play devil's advocate: I also recognize, after having written this,
> that I'm something of an "old fogey" in Emacs terms, with a special place for
> ASCII in my heart. It's quite possible that the new-and-coming generation is
> very fond of Unicode, and they will all love this feature, since it gives
> Emacs a more modern "feel". For their sake, I'm prepared to wait for the
> world's judgment before deciding that this feature is entirely terrible.

As a member of a somewhat younger generation, I can say that they look 
fine in Help buffers, manuals, and so on, although they do not look 
"modern". The curly single quotes feel more like certain tech in 
alternative history novels: shiny, could be considered cool in some very 
specific circles, but nobody uses anything like that in the modern times.

I never want to see them in my source code, though, and the benefits, 
whatever they are, do not justify the increased complexity in the 
associated facilities.

> But for the sake of my fellow dinosaurs, I need a way to turn it off! Please!
> They are like tiny knives, stabbing at my mind with their Unicode edges; or a
> plague of typeset locusts, foretelling the soon-to-come Apostroclypse.

The problem with using the "don't translate at all" option is that it 
allows you to write code that won't work for other users.

For instance, if we had such an option now, Alan wouldn't have found the 
presently discussed bug. It wouldn't not manifest with his setup, but it 
would manifest in default Emacs configuration where quote translation is 
enabled, and the majority of the users would suffer.

For that reason only, I'm against having an option like that.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[John Wiegley]

>>>>> Dmitry Gutov <dgutov@yandex.ru> writes:

> For instance, if we had such an option now, Alan wouldn't have found the
> presently discussed bug. It wouldn't not manifest with his setup, but it
> would manifest in default Emacs configuration where quote translation is
> enabled, and the majority of the users would suffer.

Hmm... these good arguments are getting in the way of my tirade.

Alright, I will let this go without a customization option, again. Alan, you
can setq `text-quoting-style' for now in your .emacs, although doing so may
get in the way of finding more bugs.

Let's see what the populace says, and how many pitchforks are raised. It's
quite possible we may revert this whole feature in 26, but let's get some
genuine public reaction before deciding the matter.

Within the Emacs developers, I now see strong votes against, some votes for
(are they as strong?), but I still don't know what the World At Large is going
to think.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Paul Eggert]

On 05/04/2016 11:57 AM, Alan Mackenzie wrote:
> By "something like it", you probably mean something
> that leaves the translation mandatory.

No, I mean something that avoids quote translation entirely, just in a 
simpler way.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Andreas Schwab]

Eli Zaretskii <eliz@gnu.org> writes:

>> > > Surely it should be a configuration variable.  Some people, like me,
>> > > don't like being lied to by a computer program.
>> 
>> > You are not being lied to.  Just think of ` and ' as yet another
>> > format specifier character, not unlike %.
>> 
>> They're not - they're literal characters.
>
> So is %.

% has always been documented as being special.  The only special
character.  Adding more special characters is an incompatible change and
breaks existing uses.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Paul Eggert]

On 05/04/2016 11:57 AM, Alan Mackenzie wrote:
>> I proposed a scheme, ....
> By which you mean you installed your changes on the master branch.
>
No, I mean that I emailed patches to bug-gnu-emacs, which is the 
documented way to suggest changes. There was a reasonably lengthy 
discussion about the patches and I eventually installed into master a 
patchset that was quite a bit different from what I originally proposed, 
after getting an OK from Emacs's then-maintainer.

At the time I didn't think it was that big a deal: it's just quoting in 
diagnostics, after all. With the benefit of hindsight I now see that I 
was mistaken about how strongly some users are attached to quoting `like 
this'. I am sorry you were not involved earlier, as that would likely 
have saved us a lot of grief. I am also sorry about the whole 
misunderstanding and will strive to publicize future changes on 
emacs-devel more prominently, even changes that I may think are not that 
important.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Lars Ingebrigtsen]

"John Wiegley" <johnw@gnu.org> writes:

> Let's see what the populace says, and how many pitchforks are raised. It's
> quite possible we may revert this whole feature in 26, but let's get some
> genuine public reaction before deciding the matter.

Well, while I don't quite understand the whole "hatin' on quotes" thing,
I think Paul's arguments makes sense.  But we should document this in a
forward-looking manner.  We should say that Emacs in the future may do
random substitutions of characters in the first string of `message' and
`format', and that the only way to get things untranslated is this:

(format "%s" foo)

And, please, then stick to that, and not start doing at-our-whim
translations of the rest of the arguments.  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 511 bytes --]

On 05/04/2016 11:39 AM, Paul Eggert wrote:
>
> I plan to look into the further change you proposed, with an eye 
> toward installing something like it (but I hope simpler)

Attached is a proposed patch to master to do this. It disables 
translation from curved quotes, which means that quote translation now 
affects only ` and ', and it documents that the text-quoting-style 
'grave' therefore suppresses all quote translation. This shortens the 
Emacs source code by forty lines or so, which is a good sign.


[-- Attachment #2: 0001-text-quoting-style-now-affects-only-and.txt --]
[-- Type: text/plain, Size: 9662 bytes --]

From 43c28fe338ec99f7f36108b7a2fab0ecd6a284c9 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Wed, 4 May 2016 16:44:37 -0700
Subject: [PATCH] 'text-quoting-style' now affects only ` and '

Change 'text-quoting-style' so that it no longer affects
formatting of curved quotes in format arguments to functions like
'message'.  In particular, when this variable's value is 'grave',
all quotes in formats are output as-is.
* doc/lispref/help.texi (Keys in Documentation):
* doc/lispref/strings.texi (Formatting Strings):
* doc/lispref/tips.texi (Documentation Tips):
* etc/NEWS:
* src/doc.c (syms_of_doc): Document this.
* lisp/help-fns.el (describe-function-1):
* src/doc.c (text_quoting_style, Fsubstitute_command_keys)
(syms_of_doc):
* src/editfns.c (styled_format): Omit now-unnecessary code.
* src/lisp.h (LEAVE_QUOTING_STYLE): Remove.
---
 doc/lispref/help.texi    |  9 ++++-----
 doc/lispref/strings.texi |  5 ++---
 doc/lispref/tips.texi    |  3 +--
 etc/NEWS                 |  6 ++++++
 lisp/help-fns.el         |  2 +-
 src/doc.c                | 41 +++++++++++------------------------------
 src/editfns.c            | 22 ----------------------
 src/lisp.h               |  3 ---
 8 files changed, 25 insertions(+), 66 deletions(-)

diff --git a/doc/lispref/help.texi b/doc/lispref/help.texi
index 58a11f2..381c27f 100644
--- a/doc/lispref/help.texi
+++ b/doc/lispref/help.texi
@@ -332,13 +332,11 @@ Keys in Documentation
 specifies @var{mapvar}'s value as the keymap for any following
 @samp{\[@var{command}]} sequences in this documentation string.
 
-@item ‘
 @itemx `
-(left single quotation mark and grave accent) both stand for a left quote.
+(grave accent) stands for a left quote.
 
-@item ’
 @itemx '
-(right single quotation mark and apostrophe) both stand for a right quote.
+(apostrophe) stands for a right quote.
 
 @item \=
 quotes the following character and is discarded; thus, @samp{\=`} puts
@@ -357,7 +355,8 @@ Keys in Documentation
 If the variable's value is @code{curve}, the style is
 @t{‘like this’} with curved single quotes.  If the value is
 @code{straight}, the style is @t{'like this'} with straight
-apostrophes.  If the value is @code{grave}, the style is @t{`like
+apostrophes.  If the value is @code{grave},
+quotes are not translated and the style is @t{`like
 this'} with grave accent and apostrophe, the standard style
 before Emacs version 25.  The default value @code{nil}
 acts like @code{curve} if curved single quotes are displayable, and
diff --git a/doc/lispref/strings.texi b/doc/lispref/strings.texi
index ce629aa..0e5f5f1 100644
--- a/doc/lispref/strings.texi
+++ b/doc/lispref/strings.texi
@@ -840,9 +840,8 @@ Formatting Strings
 @cindex curved quotes
 @cindex curly quotes
 This function acts like @code{format}, except it also converts any
-curved single quotes in @var{string} as per the value of
-@code{text-quoting-style}, and treats grave accent (@t{`}) and
-apostrophe (@t{'}) as if they were curved single quotes.  @xref{Keys
+grave accents (@t{`}) and apostrophes (@t{'}) in @var{string} as per the
+value of @code{text-quoting-style}.  @xref{Keys
 in Documentation}.
 @end defun
 
diff --git a/doc/lispref/tips.texi b/doc/lispref/tips.texi
index d12de7a..a8589df 100644
--- a/doc/lispref/tips.texi
+++ b/doc/lispref/tips.texi
@@ -679,8 +679,7 @@ Documentation Tips
 @t{'}: @t{`like-this'} rather than @t{‘like-this’}.  This
 older convention was designed for now-obsolete displays in which grave
 accent and apostrophe were mirror images.
-
-Documentation using either convention is converted to the user's
+Documentation using this convention is converted to the user's
 preferred format when it is copied into a help buffer.  @xref{Keys in
 Documentation}.
 
diff --git a/etc/NEWS b/etc/NEWS
index 21602ff..65bd44f 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -353,6 +353,12 @@ mode bindings: 'C-c @ C-a', 'C-c @ C-t', 'C-c @ C-d', and 'C-c @ C-e.'
 before running.  This is controlled by the 'grep-save-buffers'
 variable.
 
++++
+** The variable 'text-quoting-style' no longer affects the treatment
+of curved quotes in format arguments to functions like 'message' and
+'format-message'.  In particular, when this variable's value is
+'grave', all quotes in formats are output as-is.
+
 \f
 * Lisp Changes in Emacs 25.2
 
diff --git a/lisp/help-fns.el b/lisp/help-fns.el
index d1c8b2d..040152a 100644
--- a/lisp/help-fns.el
+++ b/lisp/help-fns.el
@@ -626,7 +626,7 @@ describe-function-1
             ;; Avoid asking the user annoying questions if she decides
             ;; to save the help buffer, when her locale's codeset
             ;; isn't UTF-8.
-            (unless (memq text-quoting-style '(leave straight grave))
+            (unless (memq text-quoting-style '(straight grave))
               (set-buffer-file-coding-system 'utf-8))))))))
 
 ;; Add defaults to `help-fns-describe-function-functions'.
diff --git a/src/doc.c b/src/doc.c
index 5326433..017dd17 100644
--- a/src/doc.c
+++ b/src/doc.c
@@ -704,8 +704,6 @@ text_quoting_style (void)
       ? default_to_grave_quoting_style ()
       : EQ (Vtext_quoting_style, Qgrave))
     return GRAVE_QUOTING_STYLE;
-  else if (EQ (Vtext_quoting_style, Qleave))
-    return LEAVE_QUOTING_STYLE;
   else if (EQ (Vtext_quoting_style, Qstraight))
     return STRAIGHT_QUOTING_STYLE;
   else
@@ -982,29 +980,12 @@ Otherwise, return a new string.  */)
 	  nchars++;
 	  changed = true;
 	}
-      else if (! multibyte)
-	*bufp++ = *strp++, nchars++;
       else
 	{
-	  int len;
-	  int ch = STRING_CHAR_AND_LENGTH (strp, len);
-	  if ((ch == LEFT_SINGLE_QUOTATION_MARK
-	       || ch == RIGHT_SINGLE_QUOTATION_MARK)
-	      && quoting_style != CURVE_QUOTING_STYLE
-              && quoting_style != LEAVE_QUOTING_STYLE)
-	    {
-	      *bufp++ = ((ch == LEFT_SINGLE_QUOTATION_MARK
-			  && quoting_style == GRAVE_QUOTING_STYLE)
-			 ? '`' : '\'');
-	      strp += len;
-	      changed = true;
-	    }
-	  else
-	    {
-	      do
-		*bufp++ = *strp++;
-	      while (--len != 0);
-	    }
+	  *bufp++ = *strp++;
+	  if (multibyte)
+	    while (! CHAR_HEAD_P (*strp))
+	      *bufp++ = *strp++;
 	  nchars++;
 	}
     }
@@ -1036,7 +1017,6 @@ void
 syms_of_doc (void)
 {
   DEFSYM (Qfunction_documentation, "function-documentation");
-  DEFSYM (Qleave, "leave");
   DEFSYM (Qgrave, "grave");
   DEFSYM (Qstraight, "straight");
 
@@ -1051,15 +1031,16 @@ syms_of_doc (void)
   DEFVAR_LISP ("text-quoting-style", Vtext_quoting_style,
                doc: /* Style to use for single quotes in help and messages.
 Its value should be a symbol.  It works by substituting certain single
-quotes for certain other single quotes.  This is done in help output and
-`message' output.  It is not done in `format'.
+quotes for grave accent and apostrophe.  This is done in help output
+and in functions like `message' and `format-message'.  It is not done
+in `format'.
 
-`leave' means do not do any substitutions.
 `curve' means quote with curved single quotes \\=‘like this\\=’.
 `straight' means quote with straight apostrophes \\='like this\\='.
-`grave' means quote with grave accent and apostrophe \\=`like this\\='.
-The default value nil acts like `curve' if curved single quotes are
-displayable, and like `grave' otherwise.  */);
+`grave' means quote with grave accent and apostrophe \\=`like this\\=';
+i.e., do not alter quote marks.  The default value nil acts like
+`curve' if curved single quotes are displayable, and like `grave'
+otherwise.  */);
   Vtext_quoting_style = Qnil;
 
   DEFVAR_BOOL ("internal--text-quoting-flag", text_quoting_flag,
diff --git a/src/editfns.c b/src/editfns.c
index 11a82c3..e9df09f 100644
--- a/src/editfns.c
+++ b/src/editfns.c
@@ -3974,8 +3974,6 @@ styled_format (ptrdiff_t nargs, Lisp_Object *args, bool message)
       multibyte = true;
 
   int quoting_style = message ? text_quoting_style () : -1;
-  if (quoting_style == LEAVE_QUOTING_STYLE)
-    quoting_style = -1;
 
   /* If we start out planning a unibyte result,
      then discover it has to be multibyte, we jump back to retry.  */
@@ -4454,14 +4452,6 @@ styled_format (ptrdiff_t nargs, Lisp_Object *args, bool message)
 	}
       else
 	{
-	  /* Named constants for the UTF-8 encodings of U+2018 LEFT SINGLE
-	     QUOTATION MARK and U+2019 RIGHT SINGLE QUOTATION MARK.  */
-	  enum
-	  {
-	    uLSQM0 = 0xE2, uLSQM1 = 0x80, uLSQM2 = 0x98,
-	    /* uRSQM0 = 0xE2, uRSQM1 = 0x80, */ uRSQM2 = 0x99
-	  };
-
 	  unsigned char str[MAX_MULTIBYTE_LENGTH];
 
 	  if ((format_char == '`' || format_char == '\'')
@@ -4477,18 +4467,6 @@ styled_format (ptrdiff_t nargs, Lisp_Object *args, bool message)
 	    }
 	  else if (format_char == '`' && quoting_style == STRAIGHT_QUOTING_STYLE)
 	    convsrc = "'";
-	  else if (format_char == uLSQM0 && CURVE_QUOTING_STYLE < quoting_style
-		   && multibyte_format
-		   && (unsigned char) format[0] == uLSQM1
-		   && ((unsigned char) format[1] == uLSQM2
-		       || (unsigned char) format[1] == uRSQM2))
-	    {
-	      convsrc = (((unsigned char) format[1] == uLSQM2
-			  && quoting_style == GRAVE_QUOTING_STYLE)
-			 ? "`" : "'");
-	      format += 2;
-	      memset (&discarded[format0 + 1 - format_start], 2, 2);
-	    }
 	  else
 	    {
 	      /* Copy a single character from format to buf.  */
diff --git a/src/lisp.h b/src/lisp.h
index de74a47..1fc6130 100644
--- a/src/lisp.h
+++ b/src/lisp.h
@@ -4198,9 +4198,6 @@ extern void syms_of_callproc (void);
 /* Defined in doc.c.  */
 enum text_quoting_style
   {
-    /* Leave quotes unchanged.  */
-    LEAVE_QUOTING_STYLE,
-
     /* Use curved single quotes ‘like this’.  */
     CURVE_QUOTING_STYLE,
 
-- 
2.5.5

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Andreas Schwab]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Well, while I don't quite understand the whole "hatin' on quotes" thing,
> I think Paul's arguments makes sense.  But we should document this in a
> forward-looking manner.  We should say that Emacs in the future may do
> random substitutions of characters in the first string of `message' and
> `format', and that the only way to get things untranslated is this:
>
> (format "%s" foo)

That makes it impossible to write format strings.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Alan Mackenzie]

Hello, John.

On Wed, May 04, 2016 at 11:46:58AM -0700, John Wiegley wrote:

> Hi guys,

> I appreciate the passion on this issue; I had a feeling this would come up,
> and I think the reaction is going to be far more widespread than some expect.

> As maintainer, I'll go on record by saying that I very much dislike the new
> curly quotes logic. After the reactions I've read here, I'm convinced again
> that a customization option needs to exist in 25.1, so we can easily recommend
> it to people who react as Alan has done.

:-)

[ .... ]

> At this point, I cannot undo this feature [translation of single quotes
> in help messages and message format strings], since that would
> destabilize 25.1.  However, we must have an option to conveniently
> disable it before the release happens. Of that I am now certain. Would
> someone be willing to craft a patch, please?

I committed 32f5bf0c29bbad6524f71079e4380b8156289551 to master last night
just before I read your post.  It adds the value `leave' (as in "leave
alone and don't touch") to the set valid in `text-quoting-style'.  This
might be what you're looking for, or serve as the basis for it.
`text-quoting-style' so far remains a mere Lisp variable, not a
customisable option.

> Lastly, to play devil's advocate: I also recognize, after having written this,
> that I'm something of an "old fogey" in Emacs terms, with a special place for
> ASCII in my heart. It's quite possible that the new-and-coming generation is
> very fond of Unicode, and they will all love this feature, since it gives
> Emacs a more modern "feel". For their sake, I'm prepared to wait for the
> world's judgment before deciding that this feature is entirely terrible.

We old fogies need our tools just as much as the new generations do.

[ .... ]

> -- 
> John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
> http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Eli Zaretskii]

> Cc: 23425@debbugs.gnu.org
> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Wed, 4 May 2016 16:49:12 -0700
> 
> Attached is a proposed patch to master to do this. It disables 
> translation from curved quotes, which means that quote translation now 
> affects only ` and ', and it documents that the text-quoting-style 
> 'grave' therefore suppresses all quote translation. This shortens the 
> Emacs source code by forty lines or so, which is a good sign.

Sounds good to me, thanks.  (I didn't have time to actually try this
yet.)

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Eli Zaretskii]

> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Wed, 4 May 2016 23:23:11 +0300
> Cc: eggert@cs.ucla.edu, 23425@debbugs.gnu.org
> 
>       4. It changes Emacs according to the preferences of specific people, without
>          giving others -- in the very spirit of Emacs! -- an option to choose
>          one's desired behavior in the standard fashion.
> 
>     At this point, I cannot undo this feature, since that would destabilize 25.1.
> 
> Can we actually leave it in, while all but promising to take it out in the next release?
> 
> That would seem like an odd choice.

I agree: this is not a good idea.  If we even suspect we will be
taking this out soon, we shouldn't introduce it.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Eli Zaretskii]

> From: John Wiegley <jwiegley@gmail.com>
> Cc: Eli Zaretskii <eliz@gnu.org>,  eggert@cs.ucla.edu,  23425@debbugs.gnu.org
> Date: Wed, 04 May 2016 11:46:58 -0700
> 
> As maintainer, I'll go on record by saying that I very much dislike the new
> curly quotes logic.

I'm not a big fan of them, either.

But this isn't about mine or yours or Paul's personal preferences.
This is about the Emacs users out there.  It is claimed that the
majority wants these quotes, and that Emacs using `this quoting' is
perceived as anything but modern, to put it mildly.

There are those who claim the exact opposite: that the majority wants
to remain with `this'.  I imagine there's also those who don't care
much either way, and have not spoken about this.

How can we know which way is the truth, except by letting a larger
group of users try that?  I don't see any other practical way of doing
that.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[John Wiegley]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

> How can we know which way is the truth, except by letting a larger group of
> users try that? I don't see any other practical way of doing that.

I agree, we'll need to see what the Majority actually thinks, since it's
impossible to ask them here.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 678 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> As maintainer, I'll go on record by saying that I very much dislike the new
>> curly quotes logic.
>
> I'm not a big fan of them, either.

Me neither, but...

> There are those who claim the exact opposite: that the majority wants
> to remain with `this'.  I imagine there's also those who don't care
> much either way, and have not spoken about this.

... I think most users won't even notice it (as long as it does not
appear in source code).

In fact, I know several users of Emacs who are using the pretests, and I
asked them about this.  None of them noticed it until I mentioned it,
and then told me they didn't care at all.

Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Paul Eggert]

On 05/05/2016 08:51 AM, Eli Zaretskii wrote:
> Sounds good to me, thanks.

Thanks, no further comment so I merged it into master.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Glenn Morris]

A year has passed. Is this report still relevant?

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Paul Eggert]

On 05/30/2017 06:07 PM, Glenn Morris wrote:
> A year has passed. Is this report still relevant?

No, as the patch I installed into master a year ago appears to have 
fixed the problem by supporting Alan's original request in Bug#23425#26 
of having a value for text-quoting-style that would "suppress all 
substitution of quotes". (setq text-quoting-style 'grave) now has the 
desired effect. Closing.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Alan Mackenzie]

Hello, Paul.

On Tue, May 30, 2017 at 18:42:00 -0700, Paul Eggert wrote:
> On 05/30/2017 06:07 PM, Glenn Morris wrote:
> > A year has passed. Is this report still relevant?

Yes, it is very much relevant.  The bug has not been fixed.  In
particular, somebody using `message' to generate quotes in the natural
and obvious fashion (possibly intending to resubmit them to the lisp
reader) will get them corrupted, and will lose much time and energy
attempting to diagnose the puzzling bug which results, possibly giving
up.

The only documentation for this feature is obscure and hidden.  There
is nothing in `message''s doc string which explicitly states likely
unwanted replacement of characters takes place; just a puzzling
"exception proves the rule" paragraph.

> No, as the patch I installed into master a year ago appears to have 
> fixed the problem by supporting Alan's original request in Bug#23425#26 

That appearance is deceptive.  And my original complaint was most
assuredly not in #26 of that thread.  Your workaround did not fix the
problem.  The problem, stated above, is the waste of other people's time
and energy inevitably caused by the current code and default options.  A
fix would be a state in which somebody could use `message' in emacs -Q
without getting caught up in all this stuff.

> of having a value for text-quoting-style that would "suppress all 
> substitution of quotes". (setq text-quoting-style 'grave) now has the 
> desired effect. Closing.

Well, thanks a bunch for giving me a chance to state my view before
closing this bug.  Notice that both Glenn's post and your response were
made in the middle of the night, European time, without there being
intervening European day time.

This bug has not been fixed.  That judgement is mine, as the person who
raised it in the first place.  Did you close this bug as "won't fix"?
Although not good, that would at least be better than pretending it's
been fixed and sweeping it back under the carpet.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 589 bytes --]

On 05/31/2017 02:24 PM, Alan Mackenzie wrote:
> This bug has not been fixed.

Oh, sorry, I misunderstood your earlier comments. I reopened the bug.

> The only documentation for this feature is obscure and hidden.  There
> is nothing in `message''s doc string which explicitly states likely
> unwanted replacement of characters takes place; just a puzzling
> "exception proves the rule" paragraph.
I installed the attached patch into master, which I hope makes that doc 
string sufficiently clear. I'd rather not change the Emacs default 
behavior, though, for reasons already discussed.


[-- Attachment #2: 0001-src-editfns.c-Fmessage-Improve-doc-string-Bug-23425-.patch --]
[-- Type: text/x-patch, Size: 1219 bytes --]

From 74908f08ab6a77311e4e3c688fcd354e8beb8d97 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Wed, 31 May 2017 16:22:24 -0700
Subject: [PATCH] * src/editfns.c (Fmessage): Improve doc string
 (Bug#23425#130).

---
 src/editfns.c | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/src/editfns.c b/src/editfns.c
index a51670c..89a6724 100644
--- a/src/editfns.c
+++ b/src/editfns.c
@@ -3734,11 +3734,10 @@ In batch mode, the message is printed to the standard error stream,
 followed by a newline.
 
 The first argument is a format control string, and the rest are data
-to be formatted under control of the string.  See `format-message' for
-details.
-
-Note: (message "%s" VALUE) displays the string VALUE without
-interpreting format characters like `%', `\\=`', and `\\=''.
+to be formatted under control of the string.  Percent sign (%), grave
+accent (\\=`) and apostrophe (\\=') are special in the format; see
+`format-message' for details.  To display STRING without special
+treatment, use (message "%s" STRING).
 
 If the first argument is nil or the empty string, the function clears
 any existing message; this lets the minibuffer contents show.  See
-- 
2.9.4

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Glenn Morris]

So in your view what needs to happen for this bug to be closed?
As was explained a year ago, change your initial example to (message
"%s" ...) and your issue goes away.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Alan Mackenzie]

Hello, Paul.

On Wed, May 31, 2017 at 16:26:12 -0700, Paul Eggert wrote:
> On 05/31/2017 02:24 PM, Alan Mackenzie wrote:
> > This bug has not been fixed.

> Oh, sorry, I misunderstood your earlier comments. I reopened the bug.

Thanks.

How about transferring this discussion to emacs-devel, so that more
people would be able to see it.

> > The only documentation for this feature is obscure and hidden.  There
> > is nothing in `message''s doc string which explicitly states likely
> > unwanted replacement of characters takes place; just a puzzling
> > "exception proves the rule" paragraph.

> I installed the attached patch into master, which I hope makes that doc 
> string sufficiently clear. I'd rather not change the Emacs default 
> behavior, though, for reasons already discussed.

The default behaviour has been changed already, and that change has
caused problems and will cause further problems.

I propose restoring the default to what it was in Emacs 24.  For
simplicity's sake, only % should be special, and for translating quotes
we could introduce %' and %`, such that quote translation happens only
when a %' or %` is present.

This would solve the problem in an elegant and consistent way, would
enable quote translation to happen, yet would avoid users of literal
quote characters getting caught up in unwanted quote translations.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Dmitry Gutov]

On 6/3/17 12:02 AM, Alan Mackenzie wrote:

> I propose restoring the default to what it was in Emacs 24.  For
> simplicity's sake, only % should be special, and for translating quotes
> we could introduce %' and %`, such that quote translation happens only
> when a %' or %` is present.

I recall proposing something to that effect back in one of the original 
discussions. E.g. GCC does that with its message function.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Paul Eggert]

On 06/02/2017 02:02 PM, Alan Mackenzie wrote:
> How about transferring this discussion to emacs-devel, so that more
> people would be able to see it.

We had that discussion there already, and I doubt whether it’d be 
worthwhile to repeat it. Of course you're free to take it there again if 
you like.

> The default behaviour has been changed already, and that change has
> caused problems and will cause further problems.

I expect that most of the practical problems have been shaken out 
already with the changes published in Emacs 25.1 last year, so that most 
of this cost has already been paid.

> I propose restoring the default to what it was in Emacs 24.  For
> simplicity's sake, only % should be special, and for translating quotes
> we could introduce %' and %`, such that quote translation happens only
> when a %' or %` is present.

That kind of approach was proposed way back when. It doesn’t address 
quotes in doc strings, though. More generally, a problem with this sort 
of approach is that although it simplifies ‘message’ (obviously), this 
is at the price of complicating everything else. I estimate we’d need to 
change 10,000 lines of Emacs source code to make that change to 
‘message’. (This is a just a quick estimate based on ‘grep '".*`.*"'’.) 
The resulting source code would typically be a bit harder to read than 
it is now.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Glenn Morris]

> We had that discussion there already, and I doubt whether it'd be
> worthwhile to repeat it. Of course you're free to take it there again
> if you like.

I think this is a situation where the maintainer(s) should simply
announce a decision, so that we can stop going over the same ground
again and again. (Personally I think a year and two releases means the
decision is already made.)

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Alan Mackenzie]

[ Message posted to emacs-devel@gnu.org, with Reply-To: set. ]

Hello, Paul.

On Fri, Jun 02, 2017 at 14:47:42 -0700, Paul Eggert wrote:
> On 06/02/2017 02:02 PM, Alan Mackenzie wrote:
> > How about transferring this discussion to emacs-devel, so that more
> > people would be able to see it.

> We had that discussion there already, and I doubt whether it’d be 
> worthwhile to repeat it. Of course you're free to take it there again if 
> you like.

The previous discussion was abruptly cut short, without reaching a
conclusion, with a "we're too close to a release to change anything
now".  This came with the implicit understanding the discussion could be
countinued after the release.

> > The default behaviour has been changed already, and that change has
> > caused problems and will cause further problems.

> I expect that most of the practical problems have been shaken out 
> already with the changes published in Emacs 25.1 last year, so that most 
> of this cost has already been paid.

They have not.  We have moved from a state where everybody knew what
`message' did (in Emacs 24), to one with wild special characters which
only apply sometimes, and necessitate crazy prolix formulations to work
around unwanted translations of quote characters.

`message' is currently broken.

Introducing quote translation has introduced a burden on all users of
`message'.  Seeing how poisonous and insidious this translation can be,
it makes sense to shift this burden over to the use cases where the
programmers need quote translation, and hence will be aware of it.

> > I propose restoring the default to what it was in Emacs 24.  For
> > simplicity's sake, only % should be special, and for translating quotes
> > we could introduce %' and %`, such that quote translation happens only
> > when a %' or %` is present.

> That kind of approach was proposed way back when. It doesn’t address 
> quotes in doc strings, though.

Of course it doesn't.  Doc strings have their own peculiar quoting
mechanisms which are disjoint from those of `message'.

> More generally, a problem with this sort of approach is that although
> it simplifies ‘message’ (obviously), this is at the price of
> complicating everything else.

What is the "everything else" that gets thus complicated?  I don't see
anything else getting more complicated.

> I estimate we’d need to change 10,000 lines of Emacs source code to
> make that change to ‘message’. (This is a just a quick estimate based
> on ‘grep '".*`.*"'’.) The resulting source code would typically be a
> bit harder to read than it is now.

I think this estimate is way over the mark.  There are around 17,000
occurrances of "message" in our Lisp sources, and probably a few in our
C sources.  Only (some of) those containing the quote characters in the
format string would need amendment.  These will comprise a tiny portion
of these ~17,000, and can be found easily enough with a script.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Alan Mackenzie]

[ Posted to emacs-devel@gnu.org.  Reply-To: set there too. ]

On Fri, Jun 02, 2017 at 20:13:22 -0400, Glenn Morris wrote:

> > We had that discussion there already, and I doubt whether it'd be
> > worthwhile to repeat it. Of course you're free to take it there again
> > if you like.

The discussion was abruptly suspended with "we're too close to a
release", without any conclusion having been reached.

`message' and friends are in a thoroughly unsatisfactory state, with
wild ad-hoc rules, and obstruse formulations needed as workarounds.

> I think this is a situation where the maintainer(s) should simply
> announce a decision, so that we can stop going over the same ground
> again and again. (Personally I think a year and two releases means the
> decision is already made.)

The decision was made in the near infinite number of releases prior to
Emacs 25 to have a rational and clear system of special characters
beginning with %.  Quote translation, which was introduced into Emacs
without a consensus amongst developers has destroyed that rationality
and simplicity.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Alan Mackenzie]

Hello, Glenn.

On Thu, Jun 01, 2017 at 12:56:05 -0400, Glenn Morris wrote:

> So in your view what needs to happen for this bug to be closed?
> As was explained a year ago, change your initial example to (message
> "%s" ...) and your issue goes away.

No it does not.  The issue is that this is a trap waiting to spring and
bite hackers.  Also the workaround you identify is ugly and irregular -
it would involve writing two format strings (one of them being the "%s")
to do one `message' invocation.

I have identified a regular and rational way to repair the damage,
namely to introduce the new %-constructs %` and %', which would invoke
quote translation, and would remain compatible with `message' as it was
before Emacs-25.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Alan Mackenzie]

[ Posted to emacs-devel@gnu.org, Reply-To: set there, too. ]

Hello, Dmitry.

On Sat, Jun 03, 2017 at 00:15:49 +0300, Dmitry Gutov wrote:
> On 6/3/17 12:02 AM, Alan Mackenzie wrote:

> > I propose restoring the default to what it was in Emacs 24.  For
> > simplicity's sake, only % should be special, and for translating quotes
> > we could introduce %' and %`, such that quote translation happens only
> > when a %' or %` is present.

> I recall proposing something to that effect back in one of the original 
> discussions. E.g. GCC does that with its message function.

Can you see any disadvantage with such a change?

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Dmitry Gutov]

On 6/4/17 12:10 AM, Alan Mackenzie wrote:

>> I recall proposing something to that effect back in one of the original
>> discussions. E.g. GCC does that with its message function.
> 
> Can you see any disadvantage with such a change?

IIRC, it was waved away with complaints of verbosity.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Paul Eggert]

Alan Mackenzie wrote:

> We have moved from a state where everybody knew what
> `message' did (in Emacs 24), to one with wild special characters which
> only apply sometimes, and necessitate crazy prolix formulations to work
> around unwanted translations of quote characters.

This exaggerates somewhat. We moved from Emacs 24 where only % is special, to 
Emacs 25 where %, ` and ' are special. Although some people don't know that ` 
and ' are special, that's also true for %. And although it can be annoying to 
write (message "%s" STR) to avoid unwanted translation of STR, that annoyance 
was already present for %.

> it makes sense to shift this burden over to the use cases where the
> programmers need quote translation, and hence will be aware of it.
When text-quoting-style specifies translation, most instances of ` and ' in 
Emacs messages are better off translated. So it also makes sense to translate by 
default in this situation, with a way to avoid translation in the rare cases 
where translation isn't wanted. The question is about which approach makes more 
sense, not whether one approach is sensible and the other nonsense.

>> although it simplifies ‘message’ (obviously), this is at the price of
>> complicating everything else.
> 
> What is the "everything else" that gets thus complicated?

I was referring to the hassle of going through hundreds or thousands of message 
strings or calls, deciding which instances of ` and ' should be replaced with %` 
and %', and replacing the instances accordingly. It's also possible that at 
times we'll need two format strings instead of one, complicating the code.

> There are around 17,000
> occurrances of "message" in our Lisp sources, and probably a few in our
> C sources.  Only (some of) those containing the quote characters in the
> format string would need amendment.  These will comprise a tiny portion
> of these ~17,000

How many lines do you think will be in that "tiny portion"? No matter how you 
count them, it'll be quite a few changes.

> and can be found easily enough with a script

I'm afraid not, because in many cases the string is not a simple literal 
constant argument to the message function. For starters, there's also the error 
function; that's another 14,000 text matches in the Elisp source -- many of them 
false alarms of course, but not all of them.

I'm not saying this sort of change is impossible. It's just that it'd be quite a 
bit of work, work that someone would need to volunteer to do. Is this really the 
best use of our limited resources?

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Alan Mackenzie]

Hello, Paul.

On Sun, Jun 04, 2017 at 14:01:42 -0700, Paul Eggert wrote:
> Alan Mackenzie wrote:

> > We have moved from a state where everybody knew what
> > `message' did (in Emacs 24), to one with wild special characters which
> > only apply sometimes, and necessitate crazy prolix formulations to work
> > around unwanted translations of quote characters.

> This exaggerates somewhat. We moved from Emacs 24 where only % is special, to 
> Emacs 25 where %, ` and ' are special.

Yes.  We moved from regularity (where %x, for varying x, and nothing
else was special) to a ragbag (where there are 3 special characters,
with the two "new" ones being syntactically totally different from %).

> Although some people don't know that ` and ' are special, that's also
> true for %.

No.  _Anybody_ who's used `message' knows that %s is how you print out
an arbitrary sexp.  Anybody who's used printf in C knows this, too.  It
is very easy not to know that ` and ' are special, and horribly easy to
get caught out by it, as happened to me.

> And although it can be annoying to write (message "%s" STR) to avoid
> unwanted translation of STR, that annoyance was already present for %.

It is not merely annoying, it is hideously irregular.  Having to write
(message "%s" (format "...." arg1 arg2 ....)) screams out "we didn't
think this through properly".  A call to message should only need one
format string.  The change I am proposing would achieve this.

This was never the case for %.  It is and always was trivially easy to
cause a literal % sign to be output by message, and there was never
danger of confusion in this.

We also have `format' and `format-message' which handle format strings
inconsistently.  (Yes, I know, `format-message' was introduced
deliberately to create this inconsistency, because `format' was no
longer able to cope on its own.)

> > it makes sense to shift this burden over to the use cases where the
> > programmers need quote translation, and hence will be aware of it.

> When text-quoting-style specifies translation, most instances of ` and ' in 
> Emacs messages are better off translated. So it also makes sense to translate by 
> default in this situation, with a way to avoid translation in the rare cases 
> where translation isn't wanted.

I disagree with this, of course.  Translating behind people's backs is
not a friendly thing to do.  Translation should only be done where it is
specifically specified.

> The question is about which approach makes more sense, not whether one
> approach is sensible and the other nonsense.

OK.

> >> although it simplifies ‘message’ (obviously), this is at the price of
> >> complicating everything else.

> > What is the "everything else" that gets thus complicated?

> I was referring to the hassle of going through hundreds or thousands of message 
> strings or calls, deciding which instances of ` and ' should be replaced with %` 
> and %', and replacing the instances accordingly.

Yes.  There are quite a lot, but not an unmanageable number.

> It's also possible that at times we'll need two format strings instead
> of one, complicating the code.

We need two strings instead of one at the moment, with (message "%s"
(format "..." .....)).  With %` and %' we'd only need one string in each
message invocation.  This is simplification.

Can you give an example of something which might need two strings?

> > There are around 17,000 occurrances of "message" in our Lisp
> > sources, and probably a few in our C sources.  Only (some of) those
> > containing the quote characters in the format string would need
> > amendment.  These will comprise a tiny portion of these ~17,000

> How many lines do you think will be in that "tiny portion"? No matter how you 
> count them, it'll be quite a few changes.

By searching for 

    "(\\(message\\|error\\)\\s +\\([^\"]\\|\"\\(\\\\.\\|[^\"\\]\\)*[`']\\)"

, i.e. an invocation of message or error followed by either something
which isn't a literal string, or a literal string containing ` or ', I
get 2745 matches in our Lisp sources.  There'll be a smaller number also
in our C sources.  I would have to enhance that regexp to recognise
comments, and maybe a few other things, but 2745 is a good first
approximation.

A very great number of these are "(error ..." handlers in condition-case
forms.  A great number of those remaining could be simply and
mechanically translated, for example "don't", "can't", "couldn't", etc.,
and a lot of "`%s'"s and "`%S'"s.

I estimate there will be a few hundred forms remaining which need
decision making to adapt them.  For example, where message is used in
macros, and the format string is a macro parameter.

> > and can be found easily enough with a script

> I'm afraid not, because in many cases the string is not a simple literal 
> constant argument to the message function. For starters, there's also the error 
> function; that's another 14,000 text matches in the Elisp source -- many of them 
> false alarms of course, but not all of them.

See above.

> I'm not saying this sort of change is impossible. It's just that it'd be quite a 
> bit of work, work that someone would need to volunteer to do. Is this really the 
> best use of our limited resources?

Clearly, that someone would have to be me.  The consequences of
surreptitious unwanted translation are so severe that I think this would
indeed be a good use of resources.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Paul Eggert]

On 06/05/2017 09:27 AM, Alan Mackenzie wrote:

> Can you give an example of something which might need two strings?

Suppose the same string is used for both messages and documentation. 
You're proposing %` in the former where ` is in the latter, so we would 
need distinct strings.People cut and paste between messages and doc 
strings, and it's possible there's even code that does this (I haven't 
checked). Using different quoting notations for the two kinds of strings 
will cause confusion and problems.

> 2745 is a good first approximation.

This is being optimistic. It's not just the message and error functions; 
there are others, ranging from general functions like user-error to 
more-specialized functions like icalendar--dmsg.These cannot be found 
merely by a simple text search for function names. A simple albeit 
tedious and somewhat error-prone approach would be to examine every ` 
and ' in every string and character constant (in either C or Elisp) and 
fix those intended for the message function or any of its callers. This 
is the sort of thing that I did when converting for Emacs 25.

> The consequences of surreptitious unwanted translation are so severe

It's not surreptitious: it's documented. And the consequences are not 
severe: they are a minor change to strings intended for human 
consumption, where humans can easily recover from unwanted translation, 
and where humans who don't want translation can easily turn it off.

As I understand it, you're proposing making hundreds or thousands of 
changes like this:

          (error "Can't find `%s' in %s" thing where)))
  =>      (error "Can%'t find %`%s%' in %s" thing where)))

These changes will adversely affect code readability, and this will have 
a maintenance cost in the long run -- not just to whoever makes the 
hundreds or thousands of changes, but to everybody who has to read the 
code afterwards.It's not at all clear that this cost is greater than the 
benefit of fixing some of the minor problems that you're objecting to.

> _Anybody_ who's used `message' knows

Perhaps I should introduce you to my students someday. I teach them 
elements of Emacs and I assure you that some of them do not know. 
Although I readily concede that more people know about % than ` or ', 
this doesn't alter the fact that it causes problems to support special 
characters of any sort in the message function. Elisp code needed to use 
(message "%s" STR) even before the change you're objecting to, and it'd 
still need (message "%s" STR) even if the change were reverted.

> It is not merely annoying, it is hideously irregular. Having to write
> (message "%s" (format "...." arg1 arg2 ....)) screams out "we didn't
> think this through properly".

Yes, and that's just as true for % as it is for ` and '. In hindsight, 
the message function should have had a better API. Hindsight is 
wonderful....

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Andreas Schwab]

On Jun 05 2017, Paul Eggert <eggert@cs.ucla.edu> wrote:

> Suppose the same string is used for both messages and
> documentation. You're proposing %` in the former where ` is in the latter,
> so we would need distinct strings.People cut and paste between messages
> and doc strings, and it's possible there's even code that does this (I
> haven't checked). Using different quoting notations for the two kinds of
> strings will cause confusion and problems.

That's a bad argument.  A doc string already has completely different
special characters.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Alan Mackenzie]

Hello, Paul.

On Mon, Jun 05, 2017 at 11:19:06 -0700, Paul Eggert wrote:
> On 06/05/2017 09:27 AM, Alan Mackenzie wrote:

[ .... ]

> > 2745 is a good first approximation.

> This is being optimistic. It's not just the message and error functions; 
> there are others, ranging from general functions like user-error to 
> more-specialized functions like icalendar--dmsg.These cannot be found 
> merely by a simple text search for function names.

Maybe not, but 2745 remains a good first approximation.  Other functions
which call message will be found in their turn, and the number of calls
of things like icalendar--dmsg will be small, both individually and in
total.  There are around 275 calls to message which have a non-literal
format argument.

> A simple albeit tedious and somewhat error-prone approach would be to
> examine every ` and ' in every string and character constant (in
> either C or Elisp) and fix those intended for the message function or
> any of its callers. This is the sort of thing that I did when
> converting for Emacs 25.

> > The consequences of surreptitious unwanted translation are so severe

> It's not surreptitious: it's documented.

And this documentation is useless for preventing the problems.  Somebody
using message to output Lisp will use ' just as I did - and suffer the
same horrendous problems trying to make sense of non-sensical error
messages saying 'foo does not exist, when it plainly does.  Do you,
perhaps, have another strategem for preventing this problem?

As I said, the burden of dealing with this translation should fall on
those who are actually and consciously using it.  Unless, of course,
you're in favour of hackers remaining unaware of the stealthy
alterations being made to their expressed intentions.  Such a burden, on
those wanting the translation, is small.  

> And the consequences are not severe: they are a minor change to
> strings intended for human consumption, where humans can easily
> recover from unwanted translation, and where humans who don't want
> translation can easily turn it off.

When I got caught by this bug there was no "easy" way to turn off the
translation.  The process of turning it off involved a long period of
puzzlement over the non-sensical error messages referred to above, and
an even longer period of being very angry at having my work messed up by
Emacs.  How do you propose to prevent such puzzlement and anger in the
future, if not by %` and %'?  And I was somebody who had been familiar
with discussions on emacs-devel over such matters.  Such things might
not seem severe to you.  I am sure they are to many people.

> As I understand it, you're proposing making hundreds or thousands of 
> changes like this:

>           (error "Can't find `%s' in %s" thing where)))
>   =>      (error "Can%'t find %`%s%' in %s" thing where)))

Yes.

> These changes will adversely affect code readability, and this will have 
> a maintenance cost in the long run

No.  The addition of quote translation is what has affected readability.
It is currently implicit and vague, a bit like the lack of connection
between written English and spoken English.  I am proposing restoring it
to an explicit form, where the sole special character is %, as it had
been for ever before Emacs 25.

I don't think you've considered the maintenance cost of this
impliciticity, this vagueness, the fact that people will be using ` and
' in format strings without understanding what they mean.

> -- not just to whoever makes the hundreds or thousands of changes, but
> to everybody who has to read the code afterwards.

Those people will be able to understand the code much more easily with
explicit %` and %' rather than the stealthy behind-the-back translation
which happens at the moment.

> It's not at all clear that this cost is greater than the benefit of
> fixing some of the minor problems that you're objecting to.

It's easy to minimise and dismiss other people's problems.  I don't see
non-sensical error messages as anything but a major problem.

One of the problems of the Emacs 25 approach is that the distinction
between ` and ', and the curly quotes has become vague, fuzzy, and
confused.  I do not want this confusion to carry on into Emacs 26.

> > _Anybody_ who's used `message' knows

> Perhaps I should introduce you to my students someday. I teach them 
> elements of Emacs and I assure you that some of them do not know.

The ones who have never used `message', you mean.  You're not seriously
telling me that any of your students who've written a message call with
a "%s" in the format string remain unaware of the role of %, are you?

> Although I readily concede that more people know about % than ` or ', 
> this doesn't alter the fact that it causes problems to support special 
> characters of any sort in the message function. Elisp code needed to use 
> (message "%s" STR) even before the change you're objecting to,

Did it?  When and why?

> and it'd still need (message "%s" STR) even if the change were
> reverted.

When and why?

> > It is not merely annoying, it is hideously irregular. Having to write
> > (message "%s" (format "...." arg1 arg2 ....)) screams out "we didn't
> > think this through properly".

> Yes, and that's just as true for % as it is for ` and '.

Not at all.  In message, % causes no problems.  ` and ' do.

> In hindsight, the message function should have had a better API.
> Hindsight is wonderful....

If you think a better API could have been conceived, outline it.  I
don't.  And even if it could, that's no excuse to worsen the workings of
message.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Clément Pit-Claudel]

On 2017-06-05 16:37, Alan Mackenzie wrote:
>  How do you propose to prevent such puzzlement and anger in the
> future, if not by %` and %'?

Note that this introduces a backwards compatibility issue, if not done carefully:

  Debugger entered--Lisp error: (error "Not enough arguments for format string")
    message("%`woops%'")
    eval((message "%`woops%'") nil)

Did something similar happen with \=' ?

Clément.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Paul Eggert]

 >> Elisp code needed to use
 >> (message "%s" STR) even before the change you're objecting to,
 >
 > Did it?  When and why?

Yes, because one can’t pass arbitrary strings to the message function 
and expect them to be displayed as-is. This has been true for years. For 
example, lisp/ChangeLog.8 says this:

   1999-02-23  Richard M. Stallman  <rms@gnu.org>

     * subr.el (with-temp-message): Use %s so % in old msg won't fool us.

This use of (message "%s" ...) is still in master, and would still be 
needed even if we went back to Emacs 24-style message formatting.

 > Somebody using message to output Lisp will use ' just as I did - and 
suffer the
 > same horrendous problems

Sure, like the “horrendous” problems with %. For example, in Emacs 24 
your example, when used with data involving %:

   (setq c-parse-state-state '((a '%%)))
   (message
    (concat "(setq "
     (mapconcat
      (lambda (arg)
        (format "%s %s%s" (car arg)
            (if (atom (cdr arg)) "" "'")
            (if (markerp (cdr arg))
            (format "(copy-marker %s)" (marker-position (cdr arg)))
          (cdr arg))))
      c-parse-state-state "  ")
     ")"))

returns "(setq a '((quote %)))", which is obviously wrong and which 
results in silent data corruption.

 > Do you, perhaps, have another strategem for preventing this problem?

Sure: don’t pass arbitrary strings to the message function.

 > How do you propose to prevent such puzzlement and anger in the future

Not by this:

 >>           (error "Can't find `%s' in %s" thing where)))
 >>   =>      (error "Can%'t find %`%s%' in %s" thing where)))

For Emacs code this would likely be a cure worse than the disease, by 
causing more puzzlement and anger than it would prevent. It would make 
formats significantly harder to read. And as Clément mentioned, it would 
introduce compatibility problems of its own.

There is a better way if the primary goal is to avoid quote translation:

            (error "Can't find `%s' in %s" thing where)))
    =>      (error "Can’t find ‘%s’ in %s" thing where)))

Compared to %` and %', this is simpler, easier to read, and more 
compatible with current and older Emacs versions. A downside, though, is 
that it would involve changing hundreds or thousands of strings in the 
Emacs source (just as %` and %' would).

 > You're not seriously
 > telling me that any of your students who've written a message call with
 > a "%s" in the format string remain unaware of the role of %, are you?

Sure, they learn about % after the message function doesn’t work the way 
they naively expected. In that respect, % is like ` and '.

 > There are around 275 calls to message which have a non-literal
 > format argument.

Each one stands for possibly many other calls, and we don’t know how 
many of these other calls might cause a problem.

 > The consequences of surreptitious unwanted translation ...
 >> It's not surreptitious: it's documented.
 > And this documentation is useless for preventing the problems.

True, documentation by itself does not prevent programming problems. 
However, this doesn’t change the fact that quote translation is 
documented. It is not “surreptitious” or “implicit” or “vague” or 
“stealthy” or “fuzzy”.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Andreas Schwab]

On Jun 05 2017, Paul Eggert <eggert@cs.ucla.edu> wrote:

>>> Elisp code needed to use
>>> (message "%s" STR) even before the change you're objecting to,
>>
>> Did it?  When and why?
>
> Yes, because one can’t pass arbitrary strings to the message function and
> expect them to be displayed as-is.

Another bad argument, because we are talking about format strings, which
you cannot be replaced by "%s".

>>>           (error "Can't find `%s' in %s" thing where)))
>>>   =>      (error "Can%'t find %`%s%' in %s" thing where)))

A better way would be a new modifier to place quotes around the
argument.  That's what GCC is using now.  Also, contractions like don't
should be avoided anyway.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Clément Pit-Claudel]

On 2017-06-06 04:21, Andreas Schwab wrote:
> Also, contractions like don't
> should be avoided anyway.

That doesn't matter much in the long run, I think: we intend to support languages in which apostrophes in the middle of words can't be avoided.

Clément.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Yuri Khan]

On Tue, Jun 6, 2017 at 7:24 PM, Clément Pit-Claudel
<cpitclaudel@gmail.com> wrote:
> On 2017-06-06 04:21, Andreas Schwab wrote:
>> Also, contractions like don't
>> should be avoided anyway.
>
> That doesn't matter much in the long run, I think: we intend to support languages in which apostrophes in the middle of words can't be avoided.

In that long run, I have more news for you: there are languages which
have a letter that looks almost, but not exactly, like the apostrophe
punctuation mark, but is encoded differently. (E.g. Ukrainian, U+02BC
Modifier Letter Apostrophe, as opposed to U+2019 Right Single
Quotation Mark).

Good luck substituting the correct character in each case given only '
U+0027 Apostrophe.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Clément Pit-Claudel]

On 2017-06-06 09:55, Yuri Khan wrote:
> On Tue, Jun 6, 2017 at 7:24 PM, Clément Pit-Claudel
> <cpitclaudel@gmail.com> wrote:
>> On 2017-06-06 04:21, Andreas Schwab wrote:
>>> Also, contractions like don't
>>> should be avoided anyway.
>>
>> That doesn't matter much in the long run, I think: we intend to support languages in which apostrophes in the middle of words can't be avoided.
> 
> In that long run, I have more news for you: there are languages which
> have a letter that looks almost, but not exactly, like the apostrophe
> punctuation mark, but is encoded differently. (E.g. Ukrainian, U+02BC
> Modifier Letter Apostrophe, as opposed to U+2019 Right Single
> Quotation Mark).
> 
> Good luck substituting the correct character in each case given only '
> U+0027 Apostrophe.

Do you mean that in these languages the casual way is to write ', though the formal way would be ʼ?

Clément.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Yuri Khan]

On Wed, Jun 7, 2017 at 1:21 AM, Clément Pit-Claudel
<cpitclaudel@gmail.com> wrote:

>> In that long run, I have more news for you: there are languages which
>> have a letter that looks almost, but not exactly, like the apostrophe
>> punctuation mark, but is encoded differently. (E.g. Ukrainian, U+02BC
>> Modifier Letter Apostrophe, as opposed to U+2019 Right Single
>> Quotation Mark).
>
> Do you mean that in these languages the casual way is to write ', though the formal way would be ʼ?

Hm, I haven’t thought about that. That depends.

The default XKB Ukrainian layout maps the <TLDE> key to U+0027 and
Shift+<TLDE> to U+02BC. I have no knowledge whether X11/GNU/Linux
users actually bother to press Shift.

The Ukrainian layout on Windows (Vista and above) has only U+0027.
There exist third-party typographic layouts but I assume they are only
used by people who are (1) aware of the difference, and (2) care.

So, yes, I expect that the majority of Ukrainian computer users do in
fact enter U+0027 when the correct character is U+02BC.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Clément Pit-Claudel]

On 2017-06-06 14:59, Yuri Khan wrote:
> So, yes, I expect that the majority of Ukrainian computer users do in
> fact enter U+0027 when the correct character is U+02BC.

Makes sense, thanks for clarifying.
It might be possible to enforce that translation files use the proper character (people writing translations for Emacs will be able to rely on an appropriate Emacs input method).  Can you confirm that this would sidestep the issue that you mentioned earlier?  We'd never have ' where ʼ is meant, and so automatic `'-translation wouldn't cause unexpected issues.

(The language that prompted my initial email was unsurprisingly French, which uses ' between consecutive characters (it'd be a pain to have to type "l%'arrivée d%'un email", and I don't know anyone who types in the proper "l’arrivée d’un email", except when their text editor auto-corrects).

Clément.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Paul Eggert]

On 06/06/2017 01:21 AM, Andreas Schwab wrote:
> we are talking about format strings, which you cannot be replaced by "%s".

The original bug report is about the ‘message’ function, not about 
formats in general. For that function, "%s" is the longstanding way to 
output arbitrary strings. For formats in general, one can use the 
‘format’ function (which does not translate quotes) or the 
‘format-message’ function (which does); this suffices in practice.

> contractions like don't should be avoided anyway

As a practical matter Emacs messages often have contractions and I am 
not aware of a GNU style guideline that would discourage their use. 
Plus, some uses of apostrophes in messages are for possessives, not 
contractions. Surely we should not ban contractions and possessives 
merely because our API has glitches with apostrophes.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Eli Zaretskii]

> From: Clément Pit-Claudel <cpitclaudel@gmail.com>
> Date: Tue, 6 Jun 2017 15:39:57 -0400
> Cc: Andreas Schwab <schwab@suse.de>, Paul Eggert <eggert@cs.ucla.edu>,
> 	23425@debbugs.gnu.org, Alan Mackenzie <acm@muc.de>,
> 	Emacs developers <emacs-devel@gnu.org>
> 
> (The language that prompted my initial email was unsurprisingly French, which uses ' between consecutive characters (it'd be a pain to have to type "l%'arrivée d%'un email", and I don't know anyone who types in the proper "l’arrivée d’un email", except when their text editor auto-corrects).

You should know that there are people who claim that using u+2019 in
"l’arrivée" is incorrect, and that one should write "lʼarrivée"
instead.  The rationale is that u+2019 is a punctuation character, so
it doesn't belong in the middle of a word.  There was a long
discussion about that on the Unicode list a couple of months ago.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Andreas Schwab]

On Jun 06 2017, Paul Eggert <eggert@cs.ucla.edu> wrote:

> The original bug report is about the ‘message’ function, not about formats
> in general. For that function, "%s" is the longstanding way to output
> arbitrary strings. For formats in general, one can use the ‘format’
> function (which does not translate quotes) or the ‘format-message’
> function (which does); this suffices in practice.

That doesn't make sense, since the first argument of message is a format
string.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Drew Adams]

Could you guys please possibly pick only one mailing list,
bugs or emacs-devel?

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Alan Mackenzie]

Hello, Clément.

On Mon, Jun 05, 2017 at 17:05:43 -0400, Clément Pit-Claudel wrote:
> On 2017-06-05 16:37, Alan Mackenzie wrote:
> >  How do you propose to prevent such puzzlement and anger in the
> > future, if not by %` and %'?

> Note that this introduces a backwards compatibility issue, if not done carefully:

>   Debugger entered--Lisp error: (error "Not enough arguments for format string")
>     message("%`woops%'")
>     eval((message "%`woops%'") nil)

I think it would be done carefully!  Clearly, %` and %' would be format
elements for which the count of necessary arguments would not be
incremented.  Or, more concisely, we wouldn't count them.

[ .... ]

> Clément.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Paul Eggert]

[dropping emacs-devel on Drew's request]

On 06/07/2017 12:44 AM, Andreas Schwab wrote:
> That doesn't make sense, since the first argument of message is a 
> format string.

Sorry, I’ve lost context. I wrote that (message "%s" STR) was needed 
even before Emacs 25 for arbitrary strings STR, to avoid unwanted 
interpretation of characters in STR. Alan was dubious and wanted to know 
when and why that occurred. I gave a example from 1999 and said the 
“why” was because “one can’t pass arbitrary strings to the message 
function and expect them to be displayed as-is”. You objected to this 
justification, writing “we are talking about format strings, which you 
cannot be replaced by "%s".” This objection was unclear and the 
subsequent discussion (from my point of view, anyway) has gone off the 
rails. Perhaps you could clarify the objection by giving specific code 
that exemplifies the problem you are thinking of, presumably a problem 
that cannot easily be addressed by using (message "%s" ...).

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Alan Mackenzie]

Hello, Paul.

On Mon, Jun 05, 2017 at 17:14:37 -0700, Paul Eggert wrote:
>  >> Elisp code needed to use
>  >> (message "%s" STR) even before the change you're objecting to,

>  > Did it?  When and why?

> Yes, because one can’t pass arbitrary strings to the message function 
> and expect them to be displayed as-is.

This has absolutely nothing to do with the current bug, and I'm sorry if
I've discussed the point as though it did.  I'm not suggesting arbitrary
strings should be messaged without a "%s" format string.

What I am arguing against is being required to use _two_ format strings
in one message invocation, as (message "%s" (format "..." ...)).  This
just screams out "not thought through" and is unacceptable.

>  > Somebody using message to output Lisp will use ' just as I did - and 
>  > suffer the same horrendous problems

> Sure, like the “horrendous” problems with %.

That's pure sophistry.  _Nobody_, excepting the least bright of your
students (I'm assuming here that it's not the teaching which is at
fault) will have the slightest difficulty with % in message, because it
stands out visually.  Every format construct (up to Emacs 24) started
off with %, and that simplicity and symmetry has now been destroyed.  I
propose to restore it.

By contrast, the current ` and ' are disguised format constructs which
look like plain characters.  They should become %` and %' to make them
explicit and compatible with all the other format constructs.

> For example, in Emacs 24 your example, when used with data involving
> %:

[ .... ]

That example from CC Mode, which gave rise to this bug report, did not
have and could not have had % signs.  If the reverse had been the case,
they would have been carefully and correctly coded before even trying
the thing out, because I, like everybody else here, know that %
introduces format constructs.

>  > Do you, perhaps, have another strategem for preventing this problem?

> Sure: don’t pass arbitrary strings to the message function.

In other words, just ignore the problem, and have it hit as many people
as it will, without giving them any help.  That's not very nice of you.

>  > How do you propose to prevent such puzzlement and anger in the future

> Not by this:

>  >>           (error "Can't find `%s' in %s" thing where)))
>  >>   =>      (error "Can%'t find %`%s%' in %s" thing where)))

> For Emacs code this would likely be a cure worse than the disease,
> ....

How so?  It would make the format constructs explicit, giving maximum
control to the hacker.  Or do you want to make all the curvy quote stuff
as sly and stealthy as possible, so that as few hackers as possible will
be aware of it?

> .... by causing more puzzlement and anger than it would prevent.

How could it cause puzzlement?  It's mnemonic and consistent with the
other format constructs.

> It would make formats significantly harder to read.

But only slightly.  You seem to be saying that the formats need to be
dumbed down for hackers to understand them.  I'm sure you're not right,
here.

> And as Clément mentioned, it would introduce compatibility problems of
> its own.

Rubbish!  What Clément pointed out is that the part of styled_format
which counts format constructs and argumnts would need to be modified.
This would be trivial.

> There is a better way if the primary goal is to avoid quote translation:

>             (error "Can't find `%s' in %s" thing where)))
>     =>      (error "Can’t find ‘%s’ in %s" thing where)))

> Compared to %` and %', this is simpler, easier to read, and more 
> compatible with current and older Emacs versions.

Except those characters don't appear on non-Finnish keyboards, hence are
difficult to type, and they don't display nicely on all supported
environments.  These things make that suggestion a non-starter for
current purposes.

> A downside, though, is that it would involve changing hundreds or
> thousands of strings in the Emacs source (just as %` and %' would).

There's already a volunteer to do this work, so that's not really a
downside at all.

>  > You're not seriously
>  > telling me that any of your students who've written a message call with
>  > a "%s" in the format string remain unaware of the role of %, are you?

> Sure, they learn about % after the message function doesn’t work the way 
> they naively expected. In that respect, % is like ` and '.

Only in the same sense that Emacs is like Microsoft's notepad, in that
they're both text editors.

Only the rawest of beginners will be confused by a % in a format string.
That is not the level of experience we expect of our target users.

>  > There are around 275 calls to message which have a non-literal
>  > format argument.

> Each one stands for possibly many other calls, and we don’t know how 
> many of these other calls might cause a problem.

No, but the number will be small enough for each instance to be dealt
with individually.

>  > The consequences of surreptitious unwanted translation ...
>  >> It's not surreptitious: it's documented.

It may be documented, but it's still surreptitious.  It seems intended
by its writer to be as hidden as possible, so as to make those using it
as unaware as possible of the consequences of its use.  If you don't
like %` and %', perhaps you could suggest some other way of preventing
the nature of ` and ' in format strings from being so obscure.

>  > And this documentation is useless for preventing the problems.

> True, documentation by itself does not prevent programming problems. 
> However, this doesn’t change the fact that quote translation is 
> documented. It is not “surreptitious” or “implicit” or “vague” or 
> “stealthy” or “fuzzy”.

It is most definitely implicit, in that there is no indication in the
format string that quote translation happens, it is stealthy, in that it
goes out of its way to hide its actions.  It is vague, in that a hacker
who sort of knows message will be permanently unsure which characters in
a format string are not literal.

This documentation, no matter how good it might be made, will be
insufficient to prevent hackers falling into the trap which the code
sets.  This part of Emacs is badly thought out for this reason: it's an
accident waiting to happen.  I am proposing to fix it, and have
volunteered to do the work.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Clément Pit-Claudel]

On 2017-06-07 15:13, Alan Mackenzie wrote:
> Rubbish!  What Clément pointed out is that the part of styled_format
> which counts format constructs and argumnts would need to be modified.
> This would be trivial.

Uh? What I pointed out was that new code wouldn't work with old Emacsen.

Maybe I misunderstood your proposal?  As far as I can tell, the change you propose will force me to write every `message' call like this in my packages:
 
(if (< emacs-version 26)
    (message "Hello, `world'!")
  (message "Hello, %`world%'!"))

Am I missing something?

Clément.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Clément Pit-Claudel]

On 2017-06-07 12:50, Alan Mackenzie wrote:
> Hello, Clément.
> 
> On Mon, Jun 05, 2017 at 17:05:43 -0400, Clément Pit-Claudel wrote:
>> On 2017-06-05 16:37, Alan Mackenzie wrote:
>>>  How do you propose to prevent such puzzlement and anger in the
>>> future, if not by %` and %'?
> 
>> Note that this introduces a backwards compatibility issue, if not done carefully:
> 
>>   Debugger entered--Lisp error: (error "Not enough arguments for format string")
>>     message("%`woops%'")
>>     eval((message "%`woops%'") nil)
> 
> I think it would be done carefully!  Clearly, %` and %' would be format
> elements for which the count of necessary arguments would not be
> incremented.  Or, more concisely, we wouldn't count them.

Sorry, that's not what I was talking about.  The problem I was worrying about is that these new format constructs would raise exceptions in older versions of Emacs.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Dmitry Gutov]

On 6/7/17 10:27 PM, Clément Pit-Claudel wrote:

> Uh? What I pointed out was that new code wouldn't work with old Emacsen.

That's a valid concern, and the only tangible downside of introducing %` 
and %' that I'm aware of.

> Maybe I misunderstood your proposal?  As far as I can tell, the change you propose will force me to write every `message' call like this in my packages:
>   
> (if (< emacs-version 26)
>      (message "Hello, `world'!")
>    (message "Hello, %`world%'!"))
> 
> Am I missing something?

Or wait until you only need to support Emacs 26 and newer. Not a huge 
loss, IMO. That time will come.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Alan Mackenzie]

Hello, Clément.

On Wed, Jun 07, 2017 at 15:27:02 -0400, Clément Pit-Claudel wrote:
> On 2017-06-07 15:13, Alan Mackenzie wrote:
> > Rubbish!  What Clément pointed out is that the part of styled_format
> > which counts format constructs and argumnts would need to be modified.
> > This would be trivial.

> Uh? What I pointed out was that new code wouldn't work with old Emacsen.

> Maybe I misunderstood your proposal?  As far as I can tell, the change you propose will force me to write every `message' call like this in my packages:
 
> (if (< emacs-version 26)
>     (message "Hello, `world'!")
>   (message "Hello, %`world%'!"))

> Am I missing something?

Apologies, I misunderstood you completely.

I also missed the point you are making.  I'll have to think about it.

Thanks!

> Clément.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Alan Mackenzie]

Hello again, Clément.

On Wed, Jun 07, 2017 at 15:27:02 -0400, Clément Pit-Claudel wrote:
> On 2017-06-07 15:13, Alan Mackenzie wrote:
> > Rubbish!  What Clément pointed out is that the part of styled_format
> > which counts format constructs and argumnts would need to be modified.
> > This would be trivial.

> Uh? What I pointed out was that new code wouldn't work with old Emacsen.

> Maybe I misunderstood your proposal?  As far as I can tell, the change you propose will force me to write every `message' call like this in my packages:
 
> (if (< emacs-version 26)
>     (message "Hello, `world'!")
>   (message "Hello, %`world%'!"))

That would clearly be unacceptable.

> Am I missing something?

No, I don't think so.  But it is probably feasible to put advice on
`message' in older Emacsen.  This advice would strip the % from %` and
%'.

It should be borne in mind that the handling of ` and ' by `message' in
Emacs-25 is in any case incompatible with earlier versions of Emacs.

> Clément.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Paul Eggert]

[dropping emacs-devel on Drew's request.]

On 06/07/2017 12:13 PM, Alan Mackenzie wrote:
> What I am arguing against is being required to use _two_ format 
> strings in one message invocation, as (message "%s" (format "..." ...)).

We could address this problem by defining a new function (‘memo’, say), 
that acts like ‘message’ except that it takes just one argument and does 
no format processing. That way, instead of writing (message "%s" (format 
FMT A B C)), one could write (memo (format FMT A B C)), thus avoiding 
the need for two format strings.

> _Nobody_ will have the slightest difficulty with % in message, because 
> it stands out visually.

I don’t see what “standing out visually” has to do it. If one naively 
expects (message "30%-50% done") to display "30%-50% done", one will be 
surprised/disappointed/angry/whatever when Emacs displays only "30% done".

> It may be documented, but it's still surreptitious.  It seems intended by its writer to be as hidden as possible

It’s not intended to be “surreptitious” or “hidden” or anything like 
that, and we should fix any part of the documentation that suggests 
otherwise. It would be helpful to point out any specific parts of the 
documentation needing improvement in this area.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Andreas Schwab]

If you want to force everyone to use (message "%s" (format ...)) then
there is no point in message to use format again.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Alan Mackenzie]

Hello, Paul.

On Wed, Jun 07, 2017 at 16:28:28 -0700, Paul Eggert wrote:
> On 06/07/2017 12:13 PM, Alan Mackenzie wrote:
> > What I am arguing against is being required to use _two_ format 
> > strings in one message invocation, as (message "%s" (format "..." ...)).

> We could address this problem by defining a new function (‘memo’, say), 
> that acts like ‘message’ except that it takes just one argument and does 
> no format processing. That way, instead of writing (message "%s" (format 
> FMT A B C)), one could write (memo (format FMT A B C)), thus avoiding 
> the need for two format strings.

YUCK!  So we'd have both message and memo doing basically the same
thing, with different interfaces.  Better to make them have the same
interface.  In that case, you've got two functions which are so close to
eachother, one single function would be better.....

> > _Nobody_ will have the slightest difficulty with % in message, because 
> > it stands out visually.

> I don’t see what “standing out visually” has to do it. If one naively 
> expects (message "30%-50% done") to display "30%-50% done", one will be 
> surprised/disappointed/angry/whatever when Emacs displays only "30% done".

NOBODY will write (message "30%-50% done").  They'll be writing (message
"%s%-%s% done" low high), where low is 30 and high is 50.  The
contradiction in the % is then obvious, and they can correct their
mistaken % (both of them) to %%.

> > It may be documented, but it's still surreptitious.  It seems
> > intended by its writer to be as hidden as possible

> It’s not intended to be “surreptitious” or “hidden” or anything like 
> that, ....

That may be the case, but it _seems_ very much like it.  If the
intention had actually been to be surreptitious, what more could have
been done than was actually done?  There are also other things which
reinforce this appearance of surreptition.  They should also be fixed.

> .... and we should fix any part of the documentation that suggests
> otherwise. It would be helpful to point out any specific parts of the
> documentation needing improvement in this area.

The documentation is fine.  It's the design and coding which suggest
the surreptition, and it is the design and coding which need fixing.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Paul Eggert]

On 06/08/2017 10:34 AM, Alan Mackenzie wrote:
> YUCK! So we'd have both message and memo doing basically the 
> samething, with different interfaces.

If it’s the API difference you object to, it’d be OK for the two 
functions to have the same signature. It’s no big deal.

> you've got two functions which are so close to each other, one single function would be better.....

If having one single function would mean changing hundreds or thousands 
of callers, then there is an important sense in which one single 
function would not be better.

> NOBODY will write (message "30%-50% done")

Admittedly the example was contrived, but I thought it was cool how one 
can use % to delete parts of messages, something that is much worse than 
quote restyling. More commonly people compute a message string S 
somewhere else and then call (message S), and that does have real 
problems with %, particularly in situations where an adversary has some 
control over the contents of S.

>
>> It’s not intended to be “surreptitious” or “hidden” or anything like that, ....
> That may be the case, but it _seems_ very much like it.  If the intention had actually been to be surreptitious, what more could have been done than was actually done?

Lots. We discussed and rejected more-“surreptitious” options back before 
this stuff was installed. For example, we discussed changing ‘format’ to 
behave like ‘format-message’ does now.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Alan Mackenzie]

Hello, Paul.

On Thu, Jun 08, 2017 at 13:17:39 -0700, Paul Eggert wrote:
> On 06/08/2017 10:34 AM, Alan Mackenzie wrote:
> > YUCK! So we'd have both message and memo doing basically the 
> > samething, with different interfaces.

> If it’s the API difference you object to, it’d be OK for the two 
> functions to have the same signature. It’s no big deal.

It's not OK to have two functions doing the same thing.

> > you've got two functions which are so close to each other, one
> > single function would be better.....

> If having one single function would mean changing hundreds or thousands 
> of callers, then there is an important sense in which one single 
> function would not be better.

That depends solely on whether anybody's prepared to do the work.

You're proposing changing `message' to `memo' for those callers anyway.
If you do that, we will end up with a horrible mess, with two almost
identical functions competing in the vast space of messaging where there
are no quote characters in the format string.  This would leave unsolved
the problem of format strings containing both literal quote characters
and quote characters for possible replacement by curlies.

We should be looking firstly at where we want to end up, and only later
at whether we can get there.  I still maintain that the most
satisfactory solution now available is for %` and %' to become part of
the format string, as already described.

> > NOBODY will write (message "30%-50% done")

> Admittedly the example was contrived, but I thought it was cool how one 
> can use % to delete parts of messages, something that is much worse than 
> quote restyling. More commonly people compute a message string S 
> somewhere else and then call (message S), and that does have real 
> problems with %, particularly in situations where an adversary has some 
> control over the contents of S.

Maybe, but that's a different problem.  We should not be adding to it.

> >> It’s not intended to be “surreptitious” or “hidden” or anything
> >> like that, ....
> > That may be the case, but it _seems_ very much like it.  If the
> > intention had actually been to be surreptitious, what more could
> > have been done than was actually done?

> Lots. We discussed and rejected more-“surreptitious” options back before 
> this stuff was installed. For example, we discussed changing ‘format’ to 
> behave like ‘format-message’ does now.

In effect, that's what was actually done.  `format-message' was
introduced and `message' was switched unconditionally to use it, despite
the serious problems this was bound to cause.

Let me summarise the current state of the discussion.  I have proposed
restoring message's property of every format specifier starting with %,
with every character which isn't % being a literal character.  Two new
format specifiers would be introduced, %` and %' to indicate the
substitution by the user's preferred version of left and right single
quote.

This would render format strings containing these new specifiers
unusable on Older Emacsen.  This would be worked around by providing
advice to message, to be applied by external packages using the new
specifiers.

There would be a significant amount of work amending format strings
which currently use ` and ' to use %` and %'.

I think the topic has been pretty thoroughly discussed.  Are there any
objections remaining to me implementing this change and committing it to
master?

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Drew Adams]

> This would render format strings containing these new specifiers
> unusable on Older Emacsen.  This would be worked around by providing
> advice to message, to be applied by external packages using the new
> specifiers.

Just what did you have in mind for that?

> There would be a significant amount of work amending format strings
> which currently use ` and ' to use %` and %'.

That's the problem for Emacs maintainers.  But the problem for
3rd-party code is much bigger, because it often needs to support
more than just the latest release.  Clement pointed this out well.

A user, including a Lisp user, should be able to write ' or ` and
get what s?he writes.  No behind-the-scenes application of makeup.
No prettying-up at all - you write ' and you get '.  Simple. Clear.
Lovely. Useful.  No need for high-heels.

> I think the topic has been pretty thoroughly discussed.  Are there any
> objections remaining to me implementing this change and committing it to
> master?

Sorry, but I cannot be in favor of this proposal, as I understand
it so far.

But I'm even less in favor of what's already been done: making
quote-translation the default behavior.  That's both a tragedy
and a travesty.  I cannot imagine what the maintainers were
thinking when they let this one get through.

It's fine to provide easy ways to _allow_ for different kinds of
quote translation.  But quote chars (including but not limited to
` and ') should NOT be translated by default.

By default, chars should be respected for what they are.  Emacs
should not substitute other chars just because someone working
on Emacs development considered the substitute chars prettier
or more modern.  That's nuts.  One overly zealous coder wreaked
havoc, and we'll be wrestling with the result for a long time.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Glenn Morris]

Alan Mackenzie wrote:

> I think the topic has been pretty thoroughly discussed.  Are there any
> objections remaining to me implementing this change and committing it to
> master?

I object. This is a contentious issue, and I would like the maintainers
to make an explicit decision before you change anything.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Alan Mackenzie]

Hello, Drew.

On Fri, Jun 09, 2017 at 13:17:01 -0700, Drew Adams wrote:
> > This would render format strings containing these new specifiers
> > unusable on Older Emacsen.  This would be worked around by providing
> > advice to message, to be applied by external packages using the new
> > specifiers.

> Just what did you have in mind for that?

That this advice, applied to message in Emacs <26, would strip the %
from %` and %' in format strings.  So that, for example, the format
string "%`%s%'" would get converted to "`%s'" for older Emacsen.

> > There would be a significant amount of work amending format strings
> > which currently use ` and ' to use %` and %'.

> That's the problem for Emacs maintainers.  But the problem for
> 3rd-party code is much bigger, because it often needs to support
> more than just the latest release.  Clement pointed this out well.

This is the purpose of the above advice to message.

> A user, including a Lisp user, should be able to write ' or ` and
> get what s?he writes.  No behind-the-scenes application of makeup.
> No prettying-up at all - you write ' and you get '.  Simple. Clear.
> Lovely. Useful.  No need for high-heels.

I agree with you completely, which is why I am proposing this amendment.

> > I think the topic has been pretty thoroughly discussed.  Are there any
> > objections remaining to me implementing this change and committing it to
> > master?

> Sorry, but I cannot be in favor of this proposal, as I understand
> it so far.

That puzzles me, since we seem to agree about everything else on this
issue.

> But I'm even less in favor of what's already been done: making
> quote-translation the default behavior.  That's both a tragedy
> and a travesty.  I cannot imagine what the maintainers were
> thinking when they let this one get through.

> It's fine to provide easy ways to _allow_ for different kinds of
> quote translation.  But quote chars (including but not limited to
> ` and ') should NOT be translated by default.

This is what I am proposing: a ` or ' in the format string should stand
for itself, like any other character bar %.  %` would get translated
either to itself or a curly quote, depending on the user's settings.

> By default, chars should be respected for what they are.  Emacs
> should not substitute other chars just because someone working
> on Emacs development considered the substitute chars prettier
> or more modern.  That's nuts.  One overly zealous coder wreaked
> havoc, and we'll be wrestling with the result for a long time.

I agree with you.  In message format strings, % should be _the_ special
character.  We need one.  We don't need more than one.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Drew Adams]

> That this advice, applied to message in Emacs <26, would strip the %
> from %` and %' in format strings.  So that, for example, the format
> string "%`%s%'" would get converted to "`%s'" for older Emacsen.

I suppose it's better than nothing.  But I prefer to continue to ask
that Emacs come to its senses about this whole quote-beautification
exercise.

> > A user, including a Lisp user, should be able to write ' or ` and
> > get what s?he writes.  No behind-the-scenes application of makeup.
> > No prettying-up at all - you write ' and you get '.  Simple. Clear.
> > Lovely. Useful.  No need for high-heels.
> 
> I agree with you completely, which is why I am proposing this amendment.

You're accepting being automatically dressed up in high-heels,
but you're proposing a set of overshoes that you can slip on
over them, to cancel their effect.

I'm still asking that we reverse course on the high-heels
masquerade.  It was misguided, and it seems quite unemacsy,
to me.

> > Sorry, but I cannot be in favor of this proposal, as I understand
> > it so far.
> 
> That puzzles me, since we seem to agree about everything else on this
> issue.

See above.  It may be better than nothing, but I'm still hoping
that Emacs will come to its senses about this.

> > It's fine to provide easy ways to _allow_ for different kinds of
> > quote translation.  But quote chars (including but not limited to
> > ` and ') should NOT be translated by default.
> 
> This is what I am proposing: a ` or ' in the format string should stand
> for itself, like any other character bar %.

I agree with that, of course.  It was a mistake to silently, and
by default, make `message' handle other chars specially.

> %` would get translated either to itself or a curly quote,
> depending on the user's settings.

User settings or code settings or both?  Libraries need to be
able to control the behavior by code.  And users need to be
able to control it too.  (And conflicts need to be resolved.)

So far, I intend to just bind `text-quoting-style' around code
I'm interested in giving sane behavior.  (But I'm not against
having an advice that does essentially that for `format'.)

> > By default, chars should be respected for what they are.  Emacs
> > should not substitute other chars just because someone working
> > on Emacs development considered the substitute chars prettier
> > or more modern.  That's nuts.  One overly zealous coder wreaked
> > havoc, and we'll be wrestling with the result for a long time.
> 
> I agree with you.  In message format strings, % should be _the_ special
> character.  We need one.  We don't need more than one.

And we should not have more than one.

Note that with `text-quoting-style' we already have a way to
reach inside `format' etc. and change the behavior.  But I'm
not against that - the scope can be controlled.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Paul Eggert]

On 06/09/2017 12:41 PM, Alan Mackenzie wrote:

 > It's not OK to have two functions doing the same thing.

Nobody is proposing that. Although the proposed function is similar to 
the message function, it differs in that it does not restyle quotes. 
There is lots of precedent for having two functions with similar but not 
identical behavior: for example, the error and user-error functions.

 >> If having one single function would mean changing hundreds or 
thousands of callers, then there is an important sense in which one 
single function would not be better.
 >
 > That depends solely on whether anybody's prepared to do the work.

No, it also depends on people who have to deal with the modified code 
afterwards, as the resulting formats would be harder to read. And it 
depends not only on doing the work for Emacs itself, but also for 
third-party software written in Elisp. These are significant costs.

Users should not be required to litter their code with hard-to-read 
formats like "Can%'t encode %`0x%x%' with this buffer%'s coding system" 
to get decent quoting. This would be more error-prone than what we have 
now, and although I do not object to adding support for new formats I 
would object to requiring their use in place of what we have now.

 > You're proposing changing `message' to `memo' for those callers anyway.

No, existing callers such as (message "%s" filename) would continue to 
operate as before, and would not need to be changed.

 > This would leave unsolved the problem of format strings containing 
both literal quote characters and quote characters for possible 
replacement by curlies.

I don’t see this as a significant problem in practice. In practice, 
formats typically want all quotes translated, or no quotes translated. 
If I am wrong I would like to see practical examples of the problem 
before worrying about possible solutions.  One can address the problem 
without making any changes to Emacs; whether changing Emacs would be 
helpful depends on how the problem comes up in practice.

 >> We discussed and rejected more-“surreptitious” options back before 
his stuff was installed. For example, we discussed changing ‘format’ to 
behave like ‘format-message’ does now.
 >
 > In effect, that's what was actually done.

Not at all. Had we made the more-drastic change, we would have had to 
rewrite calls like (format "\\`%s" not-word-symbol-re). This would have 
been considerably more work, and would have resulted in far more 
compatibility problems, than the change that was eventually installed.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Drew Adams]

> Although the proposed function is similar to the message
> function, it differs in that it does not restyle quotes.

It is function `message', not your proposed function `memo',
that should not "restyle" quote chars.

If you want a function that restyles quote chars then _that_
should be a new function.  Existing code - and future code -
that uses `message' should not have any such "restyling"
imposed on it.

If you had simply offered to add such a new function, I
doubt that anyone would have complained.  Seriously.

Instead, you had to impose this incompatible and specialized
prettification on the longstanding and ubiquitous, stalwart
workhorse function `message'.

That was a grave design error.  It should never have been
allowed.

> it depends not only on doing the work for Emacs itself, but also for
> third-party software written in Elisp. These are significant costs.

Really?  You are actually arguing for concern about 3rd-party
maintenance work now?  This is like DJT complaining about fake
news from others.

Where was that concern when you hijacked `message'?  Where was
your proposal to create another function, then?

> Users should not be required to litter their code with hard-to-read
> formats like "Can%'t encode %`0x%x%' with this buffer%'s coding system"
> to get decent quoting.

And now you are concerned about users?  Where was that concern
when you imposed an incompatible change on `message'?

> existing callers such as (message "%s" filename) would continue
> to operate as before, and would not need to be changed.

Where was your concern about letting `message' callers be
able to continue operating as before, before?

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Paul Eggert]

Drew Adams wrote:
> Where was that concern when you hijacked `message'?

There is no need for personal and inflammatory language like that.

That being said, I can try to describe the software engineering concerns. The 
problem with Emacs 24 message quoting had several possible solutions, none of 
them ideal. In practice the approach taken in Emacs 25 requires far fewer 
changes to existing code than the proposed quote-related format specs, and that 
is a significant win. Although quote-related format specs also have advantages, 
these advantages do not clearly outweigh that win and the proposed specs have 
further problems of their own.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Date: Fri, 09 Jun 2017 16:31:31 -0400
> Cc: Paul Eggert <eggert@cs.ucla.edu>, 23425@debbugs.gnu.org
> 
> Alan Mackenzie wrote:
> 
> > I think the topic has been pretty thoroughly discussed.  Are there any
> > objections remaining to me implementing this change and committing it to
> > master?
> 
> I object. This is a contentious issue, and I would like the maintainers
> to make an explicit decision before you change anything.

As far as things concern me, there will be no significant changes in
these facilities for at least some time, unless there's an
overwhelming agreement for such changes (which this thread and
previous ones doesn't seem to show).  We've rocked this boat enough;
IMO we should now leave it where it is for some time and let users get
used to it and bring us some feedback, before we embark on further
adventures.

John?

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Alan Mackenzie]

Hello, Paul.

On Fri, Jun 09, 2017 at 17:20:58 -0700, Paul Eggert wrote:
> Drew Adams wrote:
> > Where was that concern when you hijacked `message'?

> There is no need for personal and inflammatory language like that.

I think there is.  In a group of people who are invariably nice to each
other, there is the opportunity for rogues to exploit that niceness, and
to do deeply unpleasant things without comeback.  In this case, you are
that rogue, and it is about time somebody called you out on your
behaviour.

Before committing changes which are controversial, or which will
adversely affect others, it's expected to have a full and open
discussion on emacs-devel.  You surreptitiously committed this backward
incompatible change to `message' on 2015-08-25, without first having
initiated such a discussion.  Because of the way you did this, those
incompatibilities remained hidden until they hit another developer (me)
in a thoroughly unpleasant way some time later.

You did indeed hijack `message'.  You changed a generally useful and
universal function into one with caveats and gotchas to satisfy your own
personal preferences and, likely, a desire to impose those preferences
on Emacs users generally.

I suggest that you now apologise for what you did back in August 2015,
promising never to do the same thing again.

[ .... ]

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[David Engster]

Alan Mackenzie writes:
> I suggest that you now apologise for what you did back in August 2015,
> promising never to do the same thing again.

https://lists.gnu.org/archive/html/bug-gnu-emacs/2016-05/msg00324.html

What more do you want?

I'm not happy with that change either, but Paul has worked on Emacs for
25 years, with almost 5000 commits on record. This does not make him
infallible, but calling him a 'rogue' is just unacceptable.

-David

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Eli Zaretskii]

> From: David Engster <deng@randomsample.de>
> Date: Sat, 10 Jun 2017 12:52:44 +0200
> Cc: Paul Eggert <eggert@cs.ucla.edu>, 23425@debbugs.gnu.org
> 
> https://lists.gnu.org/archive/html/bug-gnu-emacs/2016-05/msg00324.html
> 
> What more do you want?
> 
> I'm not happy with that change either, but Paul has worked on Emacs for
> 25 years, with almost 5000 commits on record. This does not make him
> infallible, but calling him a 'rogue' is just unacceptable.

Agreed.  We all make mistakes, and should be tolerable to those made
in good faith by others.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Alan Mackenzie]

Hello, David.

On Sat, Jun 10, 2017 at 12:52:44 +0200, David Engster wrote:
> Alan Mackenzie writes:
> > I suggest that you now apologise for what you did back in August 2015,
> > promising never to do the same thing again.

> https://lists.gnu.org/archive/html/bug-gnu-emacs/2016-05/msg00324.html

> What more do you want?

An apology and an undertaking not to repeat the offense.  What you quote
is colloquially known as a "non-apology".  If you look carefully at it,
you'll see it attributes the blame to somebody else ("With the benefit
of hindsight, I now see that I was mistaken about how strongly some
users are attached to quoting `like this'") and that attribution is
somewhat insulting and one-sided, it makes no offer to fix things, and
the undertaking not to repeat is less than wholehearted ("I will
_strive_ ... to publicize ... on emacs-devel _more_ _prominently_").
Does that ostensible apology give you much confidence that Paul won't do
the same thing again?  I have my doubts.

> I'm not happy with that change either, but Paul has worked on Emacs for
> 25 years, with almost 5000 commits on record. This does not make him
> infallible, but calling him a 'rogue' is just unacceptable.

I really wish you hadn't accused me of that after snipping my text so
that people couldn't see the two together.  What I actually wrote was
"IN THIS CASE, you are that rogue", meaning in this particular instance.
I stand by that allegation.  What I didn't do was to call Paul a rogue
in general.

I'm clearly not happy with this change.  I'd be as unhappy about it if
Paul had already written 50,000 commits, or it had been written by RMS.
But, to repeat myself, I'm most unhappy that it was slipped into Emacs
bypassing the customary discussion process.

It's a non backward-compatible change to a critical bit of Emacs.
`message' was previously a logical, strong function, and it has been
turned into a messy function which sort of works most of the time.  I
don't think this is good.

> -David

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Eli Zaretskii]

> Date: Sat, 10 Jun 2017 13:01:01 +0000
> From: Alan Mackenzie <acm@muc.de>
> Cc: Paul Eggert <eggert@cs.ucla.edu>, 23425@debbugs.gnu.org
> 
> > https://lists.gnu.org/archive/html/bug-gnu-emacs/2016-05/msg00324.html
> 
> > What more do you want?
> 
> An apology and an undertaking not to repeat the offense.

It wasn't an offense.  It was a good-faith mistake.  Please accept
what the others think about this, and please don't blow this out of
proportion.

> But, to repeat myself, I'm most unhappy that it was slipped into Emacs
> bypassing the customary discussion process.

Veteran contributors -- and Paul is definitely among them -- aren't
required to wait for a discussion before they push their commits.
When to start a discussion before making a change is up to their
judgment call.  That is how Emacs development works for many years.  I
agree that in this case the change should have been discussed more
thoroughly in advance -- but that is exactly what Paul said he regrets
he hadn't done.  So the lesson has been learned, and we are now wiser
about this and similar issues that could come up in the future.

> It's a non backward-compatible change to a critical bit of Emacs.
> `message' was previously a logical, strong function, and it has been
> turned into a messy function which sort of works most of the time.

IME, it works quite well all of the time.  Where it has some
deficiencies, I'm sure we will with time improve it so it works even
better.  That's my experience in Emacs for many years.

And with that, I think it's high time to end this discussion.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Dmitry Gutov]

On 6/10/17 4:01 PM, Alan Mackenzie wrote:

> I'm clearly not happy with this change.  I'd be as unhappy about it if
> Paul had already written 50,000 commits, or it had been written by RMS.
> But, to repeat myself, I'm most unhappy that it was slipped into Emacs
> bypassing the customary discussion process.

Much as I dislike this and related changes, and the enormous timewaste 
that ensued, the change had been approved by the back-then maintainer, 
in an even more broken form (which has been improved upon somewhat 
since). Who had never voiced a significant change in his position in any 
subsequent discussions, IIRC.

So it's hard to blame Paul for the entirety of the problem.

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Alan Mackenzie]

Hello, Eli.

On Sat, Jun 10, 2017 at 16:39:33 +0300, Eli Zaretskii wrote:

> > It's a non backward-compatible change to a critical bit of Emacs.
> > `message' was previously a logical, strong function, and it has been
> > turned into a messy function which sort of works most of the time.

> IME, it [`message'] works quite well all of the time.

It does not work at all for outputting ` and '.  Ugly workarounds are
required for that.

> Where it has some deficiencies, I'm sure we will with time improve it
> so it works even better.  That's my experience in Emacs for many years.

If it doesn't get fixed now, it will never be fixed.

> And with that, I think it's high time to end this discussion.

How about fixing this bug first?

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#23425: master branch: `message' wrongly corrupts ' to curly quote.

[Eli Zaretskii]

> Date: Sun, 11 Jun 2017 10:03:47 +0000
> Cc: deng@randomsample.de, eggert@cs.ucla.edu, 23425@debbugs.gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> > IME, it [`message'] works quite well all of the time.
> 
> It does not work at all for outputting ` and '.  Ugly workarounds are
> required for that.

We will have to disagree on the ugliness part.  Modulo that, it's
quite possible to output these two characters.

> > Where it has some deficiencies, I'm sure we will with time improve it
> > so it works even better.  That's my experience in Emacs for many years.
> 
> If it doesn't get fixed now, it will never be fixed.

It will, if we the motion to do that ever gets more support, or if we
find some grave problems we are not aware today.

> > And with that, I think it's high time to end this discussion.
> 
> How about fixing this bug first?

I'm sorry, but I don't see anything that needs to be fixed at this
time.