text-quoting-style

text-quoting-style

[Stefan Monnier]

Why do we need text-quoting-style?
[ I understand that some terminals can't display those curly quotes, but
  that should be handled by some kind of display-table trick.  ]


        Stefan

Re: text-quoting-style

[David Kastrup]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Why do we need text-quoting-style?
> [ I understand that some terminals can't display those curly quotes, but
>   that should be handled by some kind of display-table trick.  ]

Display table tricks will not help with copy&paste to external
applications and with files processed by any program rather than Emacs.

-- 
David Kastrup

Re: text-quoting-style

[Paul Eggert]

Stefan Monnier wrote:
> Why do we need text-quoting-style?
> [ I understand that some terminals can't display those curly quotes, but
>    that should be handled by some kind of display-table trick.  ]

As I understand it, Alan introduced text-quoting-style because he did not want 
curved quotes in any buffer or string, even curved quotes displayed as grave 
accent and apostrophe.

It would simplify things somewhat to remove the text-quoting-style variable, and 
to have Emacs behave as it does now when text-quoting-style is nil: namely, 
generate curved quotes in buffers and strings if displayable, and generate grave 
accent and apostrophe otherwise.  That should suffice for Alan's preferences, as 
he can run Emacs in an environment where curved quotes aren't displayable, e.g., 
with LC_ALL=C in the environment.

It sounds like you may be thinking of something even simpler, though: namely, 
always generate curved quotes in strings and buffers, but display them as grave 
accent and apostrophe if curved quotes are not displayable.  I'd prefer that 
approach too, though I expect Alan would not (otherwise why introduce 
text-quoting-style?).

Re: text-quoting-style

[Eli Zaretskii]

> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Fri, 28 Aug 2015 00:06:05 -0700
> 
> It would simplify things somewhat to remove the text-quoting-style variable, and 
> to have Emacs behave as it does now when text-quoting-style is nil: namely, 
> generate curved quotes in buffers and strings if displayable, and generate grave 
> accent and apostrophe otherwise.  That should suffice for Alan's preferences, as 
> he can run Emacs in an environment where curved quotes aren't displayable, e.g., 
> with LC_ALL=C in the environment.

Telling people who don't see the Unicode quotes to run under LC_ALL=C
is not a good idea, for at least two reasons:

  . on platforms that honor LC_ALL in the environment, it will disable
    many useful Emacs features unrelated to the issue at hand

  . on platforms that don't honor LC_ALL in the environment (Windows),
    it won't have any effect

> It sounds like you may be thinking of something even simpler, though: namely, 
> always generate curved quotes in strings and buffers, but display them as grave 
> accent and apostrophe if curved quotes are not displayable.  I'd prefer that 
> approach too, though I expect Alan would not (otherwise why introduce 
> text-quoting-style?).

Given the controversy this generated, and few distinct flavors of what
people like or dislike about it, I think it's only prudent to have
this customizable.  Once Emacs 25 is released, we can see from user
feedback more clearly where do winds blow, and then decide whether
this option should be kept or modified in some sense.

Re: text-quoting-style

[Paul Eggert]

Eli Zaretskii wrote:
>> That should suffice for Alan's preferences, as
>> >he can run Emacs in an environment where curved quotes aren't displayable, e.g.,
>> >with LC_ALL=C in the environment.
> Telling people who don't see the Unicode quotes to run under LC_ALL=C
> is not a good idea, for at least two reasons:
>
>    . on platforms that honor LC_ALL in the environment, it will disable
>      many useful Emacs features unrelated to the issue at hand

Can you give an example of such a feature?  I'm not seeing the problem.

In environments that can't handle Unicode, perhaps Emacs disables some other 
features regardless of text quoting style.  If so, it should be OK to disable 
curved quote display too.

>    . on platforms that don't honor LC_ALL in the environment (Windows),
>      it won't have any effect

My suggestion was meant for Alan's environment; he is running in a GNU/Linux 
platform where LC_ALL does have the desired effect.  The suggestion wasn't meant 
for Windows users, where I assume the problem is solved in a different way and 
no suggestion is needed.

Re: text-quoting-style

[Eli Zaretskii]

> Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org
> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Fri, 28 Aug 2015 00:54:40 -0700
> 
> Eli Zaretskii wrote:
> >> That should suffice for Alan's preferences, as
> >> >he can run Emacs in an environment where curved quotes aren't displayable, e.g.,
> >> >with LC_ALL=C in the environment.
> > Telling people who don't see the Unicode quotes to run under LC_ALL=C
> > is not a good idea, for at least two reasons:
> >
> >    . on platforms that honor LC_ALL in the environment, it will disable
> >      many useful Emacs features unrelated to the issue at hand
> 
> Can you give an example of such a feature?  I'm not seeing the problem.

Anything that is decoded using locale-coding-system, or the default
values derived from it.  If the real locale is nothing like C, I
expect the result to be strings full of raw bytes.  If the real
locale's codeset is UTF-8, you might get away for a while (because
Emacs generally strives to DTRT with unibyte strings), but with any
other codeset the problem should be immediately clear.  E.g., start
Emacs in that way in such a locale, and then look at exec-path: if
there are any non-ASCII file names there, I expect you to see raw
bytes.

Also, some of the guesswork within detect-coding stuff will do wrong
things.

> In environments that can't handle Unicode, perhaps Emacs disables some other 
> features regardless of text quoting style.  If so, it should be OK to disable 
> curved quote display too.

Sorry, "disable" was not a good choice of words.  I meant "break".

> >    . on platforms that don't honor LC_ALL in the environment (Windows),
> >      it won't have any effect
> 
> My suggestion was meant for Alan's environment

But there are people who use other environments that expressed their
dislike of this change.

> The suggestion wasn't meant for Windows users, where I assume the
> problem is solved in a different way

There's no way I know of to run programs on Windows while setting the
locale to a non-default value for just that program.  The only
possibility I'm aware of is for the program itself to call 'setlocale'
or its low-level Windows equivalents.

So if we want to remove text-quoting-style (I don't think we should),
we should at least provide an Emacs command-line option that would
cause Emacs itself call 'setlocale' at startup to switch to a C
locale.  That would at least be portable, although the problems I
mentioned above with locale-coding-system derived values will still be
there.

The next logical step is to provide an option to turn off only these
quotes.  But that's almost identical to what text-quoting-style
already provides, isn't it?

> and no suggestion is needed.

??? Of course it's needed.  Unless we would like to tell those users
to get lost.

Re: text-quoting-style

[Yuri Khan]

On Fri, Aug 28, 2015 at 2:42 PM, Eli Zaretskii <eliz@gnu.org> wrote:

> There's no way I know of to run programs on Windows while setting the
> locale to a non-default value for just that program.

AppLocale.

Re: text-quoting-style

[Eli Zaretskii]

> From: Yuri Khan <yuri.v.khan@gmail.com>
> Date: Fri, 28 Aug 2015 20:39:24 +0600
> Cc: Paul Eggert <eggert@cs.ucla.edu>, Stefan Monnier <monnier@iro.umontreal.ca>, 
> 	Emacs developers <emacs-devel@gnu.org>
> 
> On Fri, Aug 28, 2015 at 2:42 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
> > There's no way I know of to run programs on Windows while setting the
> > locale to a non-default value for just that program.
> 
> AppLocale.

Doesn't work on newer Windows versions.

Re: text-quoting-style

[Paul Eggert]

Eli Zaretskii wrote:
> E.g., start
> Emacs in that way in such a locale, and then look at exec-path: if
> there are any non-ASCII file names there, I expect you to see raw
> bytes.

I'm afraid I still don't understand this sort of scenario.  Any environment in 
which exec-path has unusual characters should be able to display those 
characters; if not, it's misconfigured, and programs other than Emacs will have 
trouble too.  This is not Alan's situation, and I don't think it's the situation 
of others who have complained about curved quotes.  So I'm still not getting why 
an LC_ALL-based approach is inappropriate here.

Re: text-quoting-style

[Eli Zaretskii]

> Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org
> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Fri, 28 Aug 2015 08:00:36 -0700
> 
> Eli Zaretskii wrote:
> > E.g., start
> > Emacs in that way in such a locale, and then look at exec-path: if
> > there are any non-ASCII file names there, I expect you to see raw
> > bytes.
> 
> I'm afraid I still don't understand this sort of scenario.  Any environment in 
> which exec-path has unusual characters should be able to display those 
> characters; if not, it's misconfigured, and programs other than Emacs will have 
> trouble too.

When you set LC_ALL=C you "misconfigure" the environment.  The shell
is still running in the original locale, so the calls to getenv will
bring you the original non-ASCII characters.  (Or are you saying that
Posix systems will somehow convert the value of, say, PATH to adpat it
to the C locale?  Windows does such things, but I thought Posix
systems don't.)

When Emacs starts, it gets the values of various coding-systems,
including the one used for DECODE_SYSTEM, from the locale's codeset.
If you set LC_ALL=C, what will you get in locale-coding-system after
startup?  I don't think it will be your shell's codeset, will it?

Re: text-quoting-style

[Paul Eggert]

Eli Zaretskii wrote:
> When you set LC_ALL=C you "misconfigure" the environment.

It appears we're miscommunicating.  I'm not suggesting to set LC_ALL=C for Emacs 
only.  I'm suggesting to set LC_ALL=C everywhere.  If your display can't handle 
non-ASCII characters, then GCC, coreutils, and many other applications will 
generate what appears to be gibberish in UTF-8 locales.  The fix is simple: 
avoid UTF-8 locales if you can't display UTF-8.  This is not an issue specific 
to Emacs, or to curved single quotation marks.

Re: text-quoting-style

[Eli Zaretskii]

> Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org
> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Fri, 28 Aug 2015 08:39:49 -0700
> 
> Eli Zaretskii wrote:
> > When you set LC_ALL=C you "misconfigure" the environment.
> 
> It appears we're miscommunicating.  I'm not suggesting to set LC_ALL=C for Emacs 
> only.  I'm suggesting to set LC_ALL=C everywhere.

Including in the shell?  Then I'm sorry for my misunderstanding.  I
thought you were suggesting to invoke Emacs as "LC_ALL=C emacs".

> If your display can't handle 
> non-ASCII characters, then GCC, coreutils, and many other applications will 
> generate what appears to be gibberish in UTF-8 locales.  The fix is simple: 
> avoid UTF-8 locales if you can't display UTF-8.  This is not an issue specific 
> to Emacs, or to curved single quotation marks.

If the users who dislike these characters are prepared to set their
locale to "C", then yes, that's a solution for them.

Re: text-quoting-style

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Thu, 27 Aug 2015 22:03:01 -0400
> 
> Why do we need text-quoting-style?

For users who don't want the Unicode characters, even though their
display can show them.  Given the controversy, I think this is
justified.

Re: text-quoting-style

[Stefan Monnier]

Eli says:
> For users who don't want the Unicode characters, even though their
> display can show them.  Given the controversy, I think this is
> justified.

David says:
> Display table tricks will not help with copy&paste to external
> applications and with files processed by any program rather than Emacs.

Paul says:
> As I understand it, Alan introduced text-quoting-style because he did not
> want curved quotes in any buffer or string, even curved quotes displayed as
> grave accent and apostrophe.

IIUC, all these are guesses or hypothetical needs.  So it looks like
there is no really good reason for that config variable.

At the very least it should not be a Custom var.

And unless someone comes up with a better, first-hand reason, we should
get rid of it.


        Stefan

Re: text-quoting-style

[Alan Mackenzie]

Hello, Stefan.

On Fri, Aug 28, 2015 at 12:42:37PM -0400, Stefan Monnier wrote:
> Eli says:
> > For users who don't want the Unicode characters, even though their
> > display can show them.  Given the controversy, I think this is
> > justified.

> David says:
> > Display table tricks will not help with copy&paste to external
> > applications and with files processed by any program rather than Emacs.

> Paul says:
> > As I understand it, Alan introduced text-quoting-style because he did not
> > want curved quotes in any buffer or string, even curved quotes displayed as
> > grave accent and apostrophe.

This is abolutely correct.  :-)

> IIUC, all these are guesses or hypothetical needs.  So it looks like
> there is no really good reason for that config variable.

Paul's assertion I have just confirmed.  Unless I've missed or
misunderstood some development in the last two months or so, the need for
`text-quoting-style' still exists: if we're going to be doing clever
things to make ` and ' get converted, or appear to get converted, into
curly quotes, then a user must be able to enable/disable this conversion.

> At the very least it should not be a Custom var.

How could it be sensible not being a custom variable?  It's for user
customisation.

> And unless someone comes up with a better, first-hand reason, we should
> get rid of it.

The reason for its existence still holds.  Or is there some other
mechanism which achieves the same thing?

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: text-quoting-style

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 580 bytes --]

Alan Mackenzie wrote:
> The reason for its existence still holds.  Or is there some other
> mechanism which achieves the same thing?

I suggested the mechanism of setting the locale, which is needed anyway in an 
environment that can't display non-ASCII characters.  That is, curved quote 
characters would still be transformed to grave accent and apostrophe in the 
*Help* buffers, which I think was your main goal; it's just that the 
text-quoting-style variable wouldn't be needed.

Something like the attached change to the manual, say, with corresponding 
changes elsewhere.

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: doc.diff --]
[-- Type: text/x-diff; name="doc.diff", Size: 2983 bytes --]

diff --git a/doc/lispref/help.texi b/doc/lispref/help.texi
index 44c09a2..fdd0a32 100644
--- a/doc/lispref/help.texi
+++ b/doc/lispref/help.texi
@@ -335,18 +335,17 @@ and @samp{\=\=} puts @samp{\=} into the output.
 @strong{Please note:} Each @samp{\} must be doubled when written in a
 string in Emacs Lisp.
 
-@defvar text-quoting-style
 @cindex curved quotes
 @cindex curly quotes
-The value of this variable specifies the style used to generate text
-quotes.  If the variable's value is @code{curve}, the style is
-@t{‘like this’} with curved single quotes.  If the value is
-@code{straight}, the style is @t{'like this'} with straight
-apostrophes.  If the value is @code{grave}, the style is @t{`like
-this'} with grave accent and apostrophe.  The default value @code{nil}
-acts like @code{curve} if curved single quotes are displayable, and
-like @code{grave} otherwise.
-@end defvar
+@cindex @env{LC_ALL} environment variable
+Docstrings can represent directed single quotes with
+either curved quotes (@t{‘} and @t{’}) or with grave accent and
+apostrophe (@t{`} and @t{'}).  These are transformed to curved single
+quotes if displayable, and to grave accent and apostrophe otherwise.
+Most modern displays can represent curved quotes with no trouble; for
+the exceptions, you can run Emacs in a locale that matches your
+display, e.g., by setting @env{LC_ALL} to @samp{C} in the environment.
+@xref{Locales}.
 
 @defun substitute-command-keys string
 This function scans @var{string} for the above special sequences and
diff --git a/doc/lispref/strings.texi b/doc/lispref/strings.texi
index 08e8e87..3705546 100644
--- a/doc/lispref/strings.texi
+++ b/doc/lispref/strings.texi
@@ -818,11 +818,12 @@ if any.
 @defun format-message string &rest objects
 @cindex curved quotes
 @cindex curly quotes
-This function acts like @code{format}, except it also converts any
-curved single quotes in @var{string} as per the value of
-@code{text-quoting-style}, and treats grave accent (@t{`}) and
-apostrophe (@t{'}) as if they were curved single quotes.  @xref{Keys
-in Documentation}.
+This function acts like @code{format}, except it also replaces grave
+accents and apostrophes (@t{`} and @t{'}) in @var{string} with curved
+quotes (@t{‘} and @t{’}) if curved quotes are displayable, and
+replaces curved quotes with grave accents and apostrophes if curved
+quotes are not displayable.  This extra replacement is similar to what
+@code{substitute-command-keys} does.  @xref{Keys in Documentation}.
 @end defun
 
 @cindex @samp{%} in format
@@ -923,8 +924,7 @@ specification is unusual in that it does not use a value.  For example,
   Any other format character results in an @samp{Invalid format
 operation} error.
 
-  Here are several examples, which assume the typical
-@code{text-quoting-style} settings:
+  Here are several examples, which assume curved quotes are displayable:
 
 @example
 @group

Re: text-quoting-style

[Alan Mackenzie]

Hello, Paul.

On Fri, Aug 28, 2015 at 10:48:10AM -0700, Paul Eggert wrote:
> Alan Mackenzie wrote:
> > The reason for its existence still holds.  Or is there some other
> > mechanism which achieves the same thing?

> I suggested the mechanism of setting the locale, which is needed anyway in an 
> environment that can't display non-ASCII characters.

I think you're referring to this paragraph:

>>> It would simplify things somewhat to remove the text-quoting-style
>>> variable, and to have Emacs behave as it does now when
>>> text-quoting-style is nil: namely, generate curved quotes in buffers
>>> and strings if displayable, and generate grave accent and apostrophe
>>> otherwise.  That should suffice for Alan's preferences, as he can
>>> run Emacs in an environment where curved quotes aren't displayable,
>>> e.g., with LC_ALL=C in the environment.

That's no good to me - I need to be able to read and type Latin-1
characters (specifically, £, ä, ö, ü, ß, Ä, Ö, Ü, and probably one or
two others), yet I don't want curly quotes.

Let me emphasise that again, I don't _WANT_ curly quotes: even if
they're perfectly capable of being displayed on my environment (which
they probably are when I'm in X-Windows) I still don't want them.  We've
discussed the reasons in some depth already.

And if I change my mind suddenly, I want to be able to enable them
readily.

> That is, curved quote characters would still be transformed to grave
> accent and apostrophe in the *Help* buffers, which I think was your
> main goal; it's just that the text-quoting-style variable wouldn't be
> needed.

I think it is needed.  Fiddling around with the locale, trying to
eliminate curlies whilst retaining all wanted characters, is going to
lead to frustration, anger, and refusal to update to the latest version
of Emacs.  The locale is too blunt an instrument for this.

> Something like the attached change to the manual, say, with corresponding 
> changes elsewhere.

[ patch fragments read, but snipped.  Thanks! ]

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: text-quoting-style

[Stefan Monnier]

>> > As I understand it, Alan introduced text-quoting-style because he did not
>> > want curved quotes in any buffer or string, even curved quotes displayed as
>> > grave accent and apostrophe.
[...]
> Paul's assertion I have just confirmed.  Unless I've missed or
> misunderstood some development in the last two months or so, the need for
> `text-quoting-style' still exists: if we're going to be doing clever
> things to make ` and ' get converted, or appear to get converted, into
> curly quotes, then a user must be able to enable/disable this conversion.

That doesn't answer the question of why we need text-quoting-style.
Instead what I take from your argument is that "if we do what we do,
then we need text-quoting-style".


        Stefan

Re: text-quoting-style

[Alan Mackenzie]

Hello, Stefan.

On Sat, Aug 29, 2015 at 10:54:41AM -0400, Stefan Monnier wrote:
> >> > As I understand it, Alan introduced text-quoting-style because he did not
> >> > want curved quotes in any buffer or string, even curved quotes displayed as
> >> > grave accent and apostrophe.
> [...]
> > Paul's assertion I have just confirmed.  Unless I've missed or
> > misunderstood some development in the last two months or so, the need for
> > `text-quoting-style' still exists: if we're going to be doing clever
> > things to make ` and ' get converted, or appear to get converted, into
> > curly quotes, then a user must be able to enable/disable this conversion.

> That doesn't answer the question of why we need text-quoting-style.
> Instead what I take from your argument is that "if we do what we do,
> then we need text-quoting-style".

We're thinking at cross purposes, somehow.  I'm not clear how my anser
was inadequate.

But...  Have you read my answer to Paul,
Message-ID: <20150828182854.GC4882@acm.fritz.box>
Date: Fri, 28 Aug 2015 18:28:54 +0000
?

Let me try again: the flag text-quoting-style enables a user to control
the handling of the curly quotes in her Emacs.  In particular, it allows
her to disable these, independently of other settings, or the
environment.  Trying to control curly quotes by some setting of the
locale is inadequate.

I believe such flexibility to be needed.  The flag would not be needed
if Emacs were to continue handling its quoting in its traditional
manner.

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: text-quoting-style

[Stefan Monnier]

> Let me try again: the flag text-quoting-style enables a user to control
> the handling of the curly quotes in her Emacs.

That describes what text-quoting-style does.  Not why we need it.

> I believe such flexibility to be needed.

Here you just state that we need it with no justification.


        Stefan

Re: text-quoting-style

[Alan Mackenzie]

Hello again, Stefan.

On Sat, Aug 29, 2015 at 09:59:05PM -0400, Stefan Monnier wrote:
> > Let me try again: the flag text-quoting-style enables a user to control
> > the handling of the curly quotes in her Emacs.

> That describes what text-quoting-style does.  Not why we need it.

> > I believe such flexibility to be needed.

> Here you just state that we need it with no justification.

OK, let's try a third time.

A user should have control over the handling of curly quotes in her
Emacs.  I take that as a given, scarcely needing justification - Emacs
is, in its very essence, a configurable program, adaptable to user needs
and preferences.  Strong opinions about preferences with regard to curly
quotes have been expressed recently on emacs-devel.  This is an
indication that we need ways of configuring curly quotes.

But if justification is wanted, I think I would not ever again update my
Emacs if that meant being forced to have these curly quotes in my
buffers.  It's quite likely other people feel as strongly about it as I
do.

The current and only mechanism for controlling these curly quotes is
text-quoting-style.  Therefore, either text-quoting-style must stay, or
some other method of implementing curly control must be implemented to
replace it.

Note that the locale is not a suitable tool for controlling curlies -
when one needs a locale with more than ASCII, yet doesn't want curly
quotes, it is likely to be difficult, too difficult, to find a suitable
locale which excludes the curly quotes.

If this answer still isn't adequate, please give me solid pointers as to
what's inadequate about it, and I'll see what I can do.

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: text-quoting-style

[Stefan Monnier]

> A user should have control over the handling of curly quotes in her
> Emacs.

For starters, I reject this statement, because it is much too broad.
And since enough add-advice can take care of it already, there's no need
for any config var just to satisfy this theoretical need.

IOW, please be a lot more concrete.

> The current and only mechanism for controlling these curly quotes is
> text-quoting-style.

"Controlling these curly quotes" is very vague.

> when one needs a locale with more than ASCII, yet doesn't want curly
> quotes,

Where exactly does this hypothetical user not want curly quotes?
And why?  What is/are the concrete problems these characters cause?

E.g. text-quoting-style doesn't prevent the existence of curly quotes in
Emacs, so for some interpretation of the above it's not sufficient.
And for other interpretations the problem could be solved some other way.


        Stefan

Re: text-quoting-style

[Paul Eggert]

Stefan Monnier wrote:
> E.g. text-quoting-style doesn't prevent the existence of curly quotes in
> Emacs, so for some interpretation of the above it's not sufficient.

I think Alan was mainly annoyed by three things in the initial implementation of 
curved quotes in docstrings.

1.  They didn't display well enough on his Linux kernel console to satisfy him.

2.  It's painful for him to search for strings containing curved quotes.

3.  When he copies from *Help* and/or *Message* buffers into other buffers, he 
wants the copied text to be in grave style `like this' rather than in curve 
style ‘like this’.


Here's one possible alternative to ‘text-quoting-style’ to address these three 
problems.

(1) is fixed by using display tables to display curved quotes in grave style. 
(This has already been implemented and no change should be needed here.)

(2) is fixed by enabling character-fold-search.  This isn't the default yet, due 
to a few problems with its implementation, but these should be fixable.

(3) could be fixed by transliterating quotes when they're yanked, if the user 
sets a variable to do this transliteration.

This alternative would be superior to what we have now, because it would also 
work for info files (which aren't handled by Emacs's current scheme).

Re: text-quoting-style

[Alan Mackenzie]

Hello, Paul.

On Sun, Aug 30, 2015 at 11:29:27PM -0700, Paul Eggert wrote:
> Stefan Monnier wrote:
> > E.g. text-quoting-style doesn't prevent the existence of curly quotes in
> > Emacs, so for some interpretation of the above it's not sufficient.

> I think Alan was mainly annoyed by three things in the initial implementation of 
> curved quotes in docstrings.

> 1.  They didn't display well enough on his Linux kernel console to satisfy him.

> 2.  It's painful for him to search for strings containing curved quotes.

> 3.  When he copies from *Help* and/or *Message* buffers into other buffers, he 
> wants the copied text to be in grave style `like this' rather than in curve 
> style ‘like this’.


> Here's one possible alternative to ‘text-quoting-style’ to address these three 
> problems.

> (1) is fixed by using display tables to display curved quotes in grave style. 
> (This has already been implemented and no change should be needed here.)

No good.  Having different characters displayed from what's in the
buffer is a recipe for confusion, frustration, and anger.

> (2) is fixed by enabling character-fold-search.  This isn't the default yet, due 
> to a few problems with its implementation, but these should be fixable.

No it isn't.  character-fold-search violates the KISS principle,
certainly for me personally.  ASCII quotes and curly quotes are distinct
characters, and if I don't mind which one a search finds, I can put [`?]
into a regexp search.

> (3) could be fixed by transliterating quotes when they're yanked, if the user 
> sets a variable to do this transliteration.

I'm not sure how this could work.   Problem 3 doesn't concern me
personally, but it will definitely impact the generic user.

> This alternative would be superior to what we have now, because it would also 
> work for info files (which aren't handled by Emacs's current scheme).

What's wrong with `text-quoting-style'?  Why are you and Stefan so keen
to get rid of it?

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: text-quoting-style

[Paul Eggert]

On 08/31/2015 01:13 PM, Alan Mackenzie wrote:
> On Sun, Aug 30, 2015 at 11:29:27PM -0700, Paul Eggert wrote:
>> (1) is fixed by using display tables to display curved quotes in grave style.
>> (This has already been implemented and no change should be needed here.)
> No good.  Having different characters displayed from what's in the
> buffer is a recipe for confusion, frustration, and anger.

It's what Emacs does now, and it works well enough.  And the basic idea 
isn't a new principle: Emacs has done it for decades for many ASCII 
characters, e.g., HT.  So it is not a problem in practice.

>> (2) is fixed by enabling character-fold-search.  This isn't the default yet, due
>> to a few problems with its implementation, but these should be fixable.
> No it isn't.  character-fold-search violates the KISS principle,
> certainly for me personally.

We could implement it so that it would be quite simple: you search for 
what you see.  That should be good enough.

> Problem 3 doesn't concern me personally

Then let's make it lower priority.

>> This alternative would be superior to what we have now, because it would also
>> work for info files (which aren't handled by Emacs's current scheme).
> What's wrong with `text-quoting-style'?
>
As I wrote, text-quoting-style doesn't do anything for info files, or 
for other text files containing curved quotes.  The approach I'm 
proposing would work for these other cases too, and it would avoid the 
heavy use of translation that Stefan has expressed concern about.

Re: text-quoting-style

[Alan Mackenzie]

Hello, Paul.

On Mon, Aug 31, 2015 at 02:29:08PM -0700, Paul Eggert wrote:
> On 08/31/2015 01:13 PM, Alan Mackenzie wrote:
> > On Sun, Aug 30, 2015 at 11:29:27PM -0700, Paul Eggert wrote:
> >> (1) is fixed by using display tables to display curved quotes in grave style.
> >> (This has already been implemented and no change should be needed here.)
> > No good.  Having different characters displayed from what's in the
> > buffer is a recipe for confusion, frustration, and anger.

> It's what Emacs does now, and it works well enough.  And the basic idea 
> isn't a new principle: Emacs has done it for decades for many ASCII 
> characters, e.g., HT.  So it is not a problem in practice.

It will be a problem in practice.  It will by lying about what character
in the buffer the glyph on the screen represents.  It will be ambiguous:
seeing a ' on the screen leaves it doubtful what the corresponding
character in the buffer is.  Current uses of display tables aren't used
to confuse users.

> >> (2) is fixed by enabling character-fold-search.  This isn't the default yet, due
> >> to a few problems with its implementation, but these should be fixable.
> > No it isn't.  character-fold-search violates the KISS principle,
> > certainly for me personally.

> We could implement it so that it would be quite simple: you search for 
> what you see.  That should be good enough.

Providing what I see is what is there, yes.

> > Problem 3 doesn't concern me personally

> Then let's make it lower priority.

But don't forget people who will be cutting and pasting with tools like
gpm.

> >> This alternative would be superior to what we have now, because it would also
> >> work for info files (which aren't handled by Emacs's current scheme).
> > What's wrong with `text-quoting-style'?

> As I wrote, text-quoting-style doesn't do anything for info files, or 
> for other text files containing curved quotes.

That's an argument for an additional facility for Info, not one for
getting rid of `text-quoting-style', unless the replacement can do the
job of `text-quoting-style'.

> The approach I'm proposing would work for these other cases too, and
> it would avoid the heavy use of translation that Stefan has expressed
> concern about.

I'm not quite up to date with the status of these things.  But I'll be
quite happy if the following hold:
(i) Quotes, apostrophes, and so on, in all our sources will continue to
  be ASCII.
(ii) Emacs can be configured such that the quotes in help buffers,
  messages, etc. are ASCII quotes.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: text-quoting-style

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 1114 bytes --]

Alan Mackenzie wrote:
>> It's what Emacs does now, and it works well enough.  And the basic idea
>> >isn't a new principle: Emacs has done it for decades for many ASCII
>> >characters, e.g., HT.  So it is not a problem in practice.
> It will be a problem in practice.  It will by lying about what character
> in the buffer the glyph on the screen represents.

Again, there's nothing new here: Emacs has been "lying" in that way for decades 
for HT and for several other ASCII characters, and it works in practice.

>   It will be ambiguous:

That's easily enough fixed.  I installed the attached patch, which uses shadow 
glyphs for quote substitutions in ASCII-only displays.  If you prefer underline 
or some other glyph face on your console please feel free to change the code.

>> text-quoting-style doesn't do anything for info files, or
>> for other text files containing curved quotes.
>
> That's an argument for an additional facility for Info

The additional facility I proposed would work for info, and would also work for 
*Help* buffers and diagnostics and would render text-quoting-style unnecessary.

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Display-replacement-quotes-with-shadow-glyphs.patch --]
[-- Type: text/x-diff; name="0001-Display-replacement-quotes-with-shadow-glyphs.patch", Size: 1378 bytes --]

From 1a3518e7c361a9ceaa017c1334a83d14e0651a4e Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Tue, 1 Sep 2015 09:18:12 -0700
Subject: [PATCH] Display replacement quotes with shadow glyphs

* lisp/startup.el (command-line): When displaying ASCII
replacements for curved quotes, use a shadow glyph instead of a
regular one, to avoid ambiguity.
---
 lisp/startup.el | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/lisp/startup.el b/lisp/startup.el
index 8c63ed2..3e29739 100644
--- a/lisp/startup.el
+++ b/lisp/startup.el
@@ -1018,11 +1018,12 @@ please check its value")
     (setq no-blinking-cursor t))
 
   ;; If curved quotes don't work, display ASCII approximations.
-  (dolist (char-repl '((?‘ . [?\`]) (?’ . [?\']) (?“ . [?\"]) (?” . [?\"])))
+  (dolist (char-repl '((?‘ . ?\`) (?’ . ?\') (?“ . ?\") (?” . ?\")))
     (when (not (char-displayable-p (car char-repl)))
       (or standard-display-table
           (setq standard-display-table (make-display-table)))
-      (aset standard-display-table (car char-repl) (cdr char-repl))))
+      (aset standard-display-table (car char-repl)
+            (vector (make-glyph-code (cdr char-repl) 'shadow)))))
   (setq internal--text-quoting-flag t)
 
   ;; Re-evaluate predefined variables whose initial value depends on
-- 
2.1.4

Re: text-quoting-style

[David Kastrup]

Paul Eggert <eggert@cs.ucla.edu> writes:

> Alan Mackenzie wrote:
>>> It's what Emacs does now, and it works well enough.  And the basic idea
>>> >isn't a new principle: Emacs has done it for decades for many ASCII
>>> >characters, e.g., HT.  So it is not a problem in practice.
>> It will be a problem in practice.  It will by lying about what character
>> in the buffer the glyph on the screen represents.
>
> Again, there's nothing new here: Emacs has been "lying" in that way
> for decades for HT and for several other ASCII characters, and it
> works in practice.

Tabs do copy&paste&search well even if their display form is a group of
spaces.  Again, we are not talking about a mere change in display here
but of actual buffer/string content.

-- 
David Kastrup

Re: text-quoting-style

[Alan Mackenzie]

Hello, Paul

On Tue, Sep 01, 2015 at 09:27:18AM -0700, Paul Eggert wrote:
> Alan Mackenzie wrote:
> >> It's what Emacs does now, and it works well enough.  And the basic idea
> >> >isn't a new principle: Emacs has done it for decades for many ASCII
> >> >characters, e.g., HT.  So it is not a problem in practice.
> > It will be a problem in practice.  It will by lying about what character
> > in the buffer the glyph on the screen represents.

> Again, there's nothing new here: Emacs has been "lying" in that way for decades 
> for HT and for several other ASCII characters, and it works in practice.

Pure sophistry.

> >   It will be ambiguous: seeing a ' on the screen leaves it doubtful
> >   what the corresponding character in the buffer is.  Current uses of
> >   display tables aren't used to confuse users.

> That's easily enough fixed.  I installed the attached patch, which uses shadow 
> glyphs for quote substitutions in ASCII-only displays.

I don't know what shadow glyphs are, and don't particularly care - they're
irrelevant.  As I've said more times than I can count, the problem is
that curly quotes are used in help buffers, messages, etc., not the way
they get displayed.  I solved that problem by introducing
`text-quoting-style', and you have reintroduced the problem by removing
`text-quoting-style'.

Please come up with an actual solution, not an irrelevance.  Or else put
`text-quoting-style' back again.

> If you prefer underline or some other glyph face on your console please
> feel free to change the code.

> >> text-quoting-style doesn't do anything for info files, or
> >> for other text files containing curved quotes.

> > That's an argument for an additional facility for Info not one for
> > getting rid of `text-quoting-style', unless the replacement can do
> > the job of `text-quoting-style'.

> The additional facility I proposed would work for info, and would also work for 
> *Help* buffers and diagnostics

For what value of "work"?  Will it enable the quoting behaviour to be
configured to be the same as in Emacs-24.5?  It not, it's no good.

>and would render text-quoting-style unnecessary.

`text-quoting-style' is necessary.  Please put it back, or replace it
with something which works properly.

[ patch read and snipped ].

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: text-quoting-style

[Paul Eggert]

Alan Mackenzie wrote:

> I solved that problem by introducing
> `text-quoting-style', and you have reintroduced the problem by removing
> `text-quoting-style'.

Not true.  I did not remove that variable.  It's still there in master, and it 
still causes Emacs to generate ASCII quotes upon request.  Your message seems to 
be based on a misconception.

Re: text-quoting-style

[Alan Mackenzie]

Hello, Paul.

On Tue, Sep 01, 2015 at 04:58:59PM -0700, Paul Eggert wrote:
> Alan Mackenzie wrote:

> > I solved that problem by introducing
> > `text-quoting-style', and you have reintroduced the problem by removing
> > `text-quoting-style'.

> Not true.  I did not remove that variable.  It's still there in master, and it 
> still causes Emacs to generate ASCII quotes upon request.  Your message seems to 
> be based on a misconception.

OK, sorry about that, I did indeed misunderstand.  Can you further
confirm that there is no intention to remove this facility?

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: text-quoting-style

[Paul Eggert]

Alan Mackenzie wrote:
> Hello, Paul.
>
> On Tue, Sep 01, 2015 at 04:58:59PM -0700, Paul Eggert wrote:
>> Alan Mackenzie wrote:
>
>>> I solved that problem by introducing
>>> `text-quoting-style', and you have reintroduced the problem by removing
>>> `text-quoting-style'.
>
>> Not true.  I did not remove that variable.  It's still there in master, and it
>> still causes Emacs to generate ASCII quotes upon request.  Your message seems to
>> be based on a misconception.
>
> OK, sorry about that, I did indeed misunderstand.  Can you further
> confirm that there is no intention to remove this facility?

I have no intention to remove it now.  Stefan is concerned about the current 
design, and as I understand it would prefer a more-general approach that would 
render text-quoting-style unnecessary.  If that happens, then of course 
text-quoting-style could be removed, as it's never appeared in a release.  But I 
don't see us removing text-quoting-style without replacing it with something else.

Re: text-quoting-style

[Eli Zaretskii]

> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Tue, 1 Sep 2015 09:27:18 -0700
> Cc: emacs-devel@gnu.org
> 
> >   It will be ambiguous:
> 
> That's easily enough fixed.  I installed the attached patch, which uses shadow 
> glyphs for quote substitutions in ASCII-only displays.

Shouldn't the same be done in w32console.el?

Re: text-quoting-style

[Paul Eggert]

On 09/01/2015 11:44 AM, Eli Zaretskii wrote:
> Shouldn't the same be done in w32console.el?

Yes, sorry, I didn't know w32console.el was duplicating that bit of 
code.  Fixed in master commit 183ad9d59757314f93aed8a9fe512c8fb48a2ed1, 
which also refactors to avoid the duplication, to make this problem less 
likely in the future.

Re: text-quoting-style

[David Kastrup]

Paul Eggert <eggert@cs.ucla.edu> writes:

> On 08/31/2015 01:13 PM, Alan Mackenzie wrote:
>> On Sun, Aug 30, 2015 at 11:29:27PM -0700, Paul Eggert wrote:
>>> (1) is fixed by using display tables to display curved quotes in grave style.
>>> (This has already been implemented and no change should be needed here.)
>> No good.  Having different characters displayed from what's in the
>> buffer is a recipe for confusion, frustration, and anger.
>
> It's what Emacs does now, and it works well enough.  And the basic
> idea isn't a new principle: Emacs has done it for decades for many
> ASCII characters, e.g., HT.  So it is not a problem in practice.

HT works fine for copy&paste&search&replace.  But characters like ’ and
‘ don't magically turn back into ' and ` if you try pasting them into
some source code or want to search for them.  If we were talking about
display table hacks and/or the display text property, this might become
the case.

But at the current point of time, the conversion of ` and ' into ‘ and ’
is much more sticky than the expansion of tabs to spaces which is a mere
display feature.

Reaching a technical consensus is not feasible as long as people keep
twisting the facts since obviously people with a different desired
outcome will twist them into different directions.

-- 
David Kastrup

Re: text-quoting-style

[Paul Eggert]

David Kastrup wrote:
> HT works fine for copy&paste&search&replace.

Yes, it “works fine”, in the sense that we‘re used to its behavior.  In that 
behavior, HT does not always stand exactly for itself; quite the contrary. 
(Even SP does not do that.)  Insisting on philosophical grounds that every 
character must always stand exactly for itself would make Emacs less useful.

Re: text-quoting-style

[David Kastrup]

Paul Eggert <eggert@cs.ucla.edu> writes:

> David Kastrup wrote:
>> HT works fine for copy&paste&search&replace.
>
> Yes, it “works fine”, in the sense that we‘re used to its behavior.
> In that behavior, HT does not always stand exactly for itself; quite
> the contrary. (Even SP does not do that.)  Insisting on philosophical
> grounds that every character must always stand exactly for itself
> would make Emacs less useful.

The curved quotes are not about making Emacs more useful.  They are
about reading aesthetics, but their effects are not restricted to the
read form.

-- 
David Kastrup

Re: text-quoting-style

[Paul Eggert]

On 09/01/2015 10:08 AM, David Kastrup wrote:
> The curved quotes are not about making Emacs more useful.  They are
> about reading aesthetics, but their effects are not restricted to the
> read form.

Exactly as with HT and SP.

Re: text-quoting-style

[David Kastrup]

Paul Eggert <eggert@cs.ucla.edu> writes:

> On 09/01/2015 10:08 AM, David Kastrup wrote:
>> The curved quotes are not about making Emacs more useful.  They are
>> about reading aesthetics, but their effects are not restricted to the
>> read form.
>
> Exactly as with HT and SP.

You keep saying that, but Emacs cuts&pastes HT and SP perfectly well
without modification.  A TAB copies and pastes as a TAB into Emacs and
any other application.  Even though its visual form tends to be a bunch
of spaces (short of calling whitespace-mode or other).

-- 
David Kastrup

Re: text-quoting-style

[Paul Eggert]

David Kastrup wrote:
> Emacs cuts&pastes HT and SP perfectly well
> without modification.

Sure, but that doesn't mean HT and SP always act as themselves and display as 
themselves; they don't.  If you search by typing C-s followed SP, for example, 
the search matches HT in the buffer.  And if you type HT into the *scratch* 
buffer first thing, the HT is ignored.  There are many other examples.  These 
behaviors are OK even though they don't follow the design philosophy that every 
character must stand for itself and nothing else.  Emacs does not follow that 
philosophy, and never has.

Re: text-quoting-style

[David Kastrup]

Paul Eggert <eggert@cs.ucla.edu> writes:

> David Kastrup wrote:
>> Emacs cuts&pastes HT and SP perfectly well
>> without modification.
>
> Sure, but that doesn't mean HT and SP always act as themselves and
> display as themselves; they don't.  If you search by typing C-s
> followed SP, for example, the search matches HT in the buffer.  And if
> you type HT into the *scratch* buffer first thing, the HT is ignored.
> There are many other examples.  These behaviors are OK even though
> they don't follow the design philosophy that every character must
> stand for itself and nothing else.  Emacs does not follow that
> philosophy, and never has.

I have my problems making a guess whether you you regard your arguments
as sound rather than the inexhaustible stream of non-sequiturs they are
or your readers as morons on which coherent thought would be wasted.

-- 
David Kastrup

Re: text-quoting-style

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > The curved quotes are not about making Emacs more useful.  They are
  > > about reading aesthetics, but their effects are not restricted to the
  > > read form.

  > Exactly as with HT and SP.

All displays and all keyboards support HT and SP, because those
chars have existed since the 1960s.

If the situation with curly quotes were indeed "exactly" the same,
no one would have any difficulty with them.


-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: text-quoting-style

[Alan Mackenzie]

Hello, Stefan.

On Sun, Aug 30, 2015 at 09:30:53PM -0400, Stefan Monnier wrote:
> > A user should have control over the handling of curly quotes in her
> > Emacs.

> For starters, I reject this statement, because it is much too broad.
> And since enough add-advice can take care of it already, there's no need
> for any config var just to satisfy this theoretical need.

> IOW, please be a lot more concrete.

I DO NOT WANT CURLY QUOTES IN MY EMACS!!!!!

How much more concrete can I get?

Are you trying to get me to swear?  How many times are you going to keep
asking the same question, snipping my answer, then asking me again?

I'm not interested in having people tell me how stupid I am for not
wanting curly quotes, I'm not interested in workarounds to work around
them, I simply don't want them on my Emacs.  An Emacs which forces them
on me is one I will not upgrade to.

> > The current and only mechanism for controlling these curly quotes is
> > text-quoting-style.

> "Controlling these curly quotes" is very vague.

OK.  Maybe you could suggest a less vague formulation.  You know what
text-quoting-style does, so why the complaint about "controlling these
curly quotes"?

> > when one needs a locale with more than ASCII, yet doesn't want curly
> > quotes,

> Where exactly does this hypothetical user not want curly quotes?
> And why?  What is/are the concrete problems these characters cause?

That hypothetical user is me.  And N. Jackson.  And Drew.  And RMS.  And
likely a lot of people who use Linux virtual terminals, or any of myriad
other systems which aren't X Windows.  And lots of X Windows users, too.
The place where we don't want curly quotes is Emacs.

These characters don't display properly on a typical LVT, you can't type
curly quotes properly on any standard keyboard layout.  There are
workarounds, sure, but I'm not interested in workarounds.  I want an
sleek efficient editing environment, and one where I have to stop and
think every time I come to a quote character, then go through the
rigmarole of C-x 8 whatever, isn't in that category.

It's a bit like asking what concrete problems transient-mark-mode
causes.  Some people just DO NOT WANT IT.  FULL STOP.

> E.g. text-quoting-style doesn't prevent the existence of curly quotes in
> Emacs, so for some interpretation of the above it's not sufficient.
> And for other interpretations the problem could be solved some other way.

Maybe it's sufficient, and if not, maybe it can be made sufficient.

But text-quoting-style is needed, otherwise we're just telling people
like me, like N. Jackson, like Drew, and like RMS that they can go and
jump in a lake.  And text-quoting-style MUST be customisable, so that
people can easily find it.

What is the thinking behind the suggestion that text-quoting-style not
be customisable?

And what do you mean by "could be solved some other way"?  When it comes
to disabling an unwanted feature, what other way is there but a
customisation variable or equivalent like M-x disable-curly-quotes?

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: text-quoting-style

[Stefan Monnier]

> I DO NOT WANT CURLY QUOTES IN MY EMACS!!!!!
> How much more concrete can I get?

Then I guess you should go back to Emacs-19, since AFAIK curly quotes
have existed one way or another in Emacs ever since Emacs-20.

So, as far as I can tell, there's nothing we can to do help you.
And text-quoting-style is no solution for you either, since Emacs's
source code and many other documents will still come with those pesky
thingies.

IOW you agree that we should get rid of text-quoting-style.


        Stefan

Re: text-quoting-style

[Dmitry Gutov]

On 09/01/2015 06:31 AM, Stefan Monnier wrote:
> since Emacs's source code

That's a choice (and a recent one), not an inevitability.

Re: text-quoting-style

[Oleh Krehel]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 09/01/2015 06:31 AM, Stefan Monnier wrote:
>> since Emacs's source code
>
> That's a choice (and a recent one), not an inevitability.

I really hope we can still get rid of the curlies in the source code.
Source code should be search-able, and not only by isearch. Also git
grep, ack, ag, and whatever new tool may come up in the future.

Re: text-quoting-style

[Alan Mackenzie]

Hello, Stefan.

First of all, sorry that my last post was a bit irritable.

On Mon, Aug 31, 2015 at 11:31:20PM -0400, Stefan Monnier wrote:
> > I DO NOT WANT CURLY QUOTES IN MY EMACS!!!!!
> > How much more concrete can I get?

> Then I guess you should go back to Emacs-19, since AFAIK curly quotes
> have existed one way or another in Emacs ever since Emacs-20.

I find the state of quoting as of Emacs 24.5 to be quite acceptable.

> So, as far as I can tell, there's nothing we can to do help you.
> And text-quoting-style is no solution for you either, since Emacs's
> source code and many other documents will still come with those pesky
> thingies.

It's an Emacs tradition that controversial new features are optional.
Curly quotes is such a new feature.  Can you assure us all that this
tradition will be upheld with respect to curly quotes?

> IOW you agree that we should get rid of text-quoting-style.

If there will be a configuration mechanism to replace it which is
capable of configuring Emacs such that the quoting behaviour is
unchanged from 24.5, then I agree it can be got rid of.

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: text-quoting-style

[Stefan Monnier]

> If there will be a configuration mechanism to replace it which is
> capable of configuring Emacs such that the quoting behaviour is
> unchanged from 24.5, then I agree it can be got rid of.

The quoting behavior will be changed.


        Stefan

Re: text-quoting-style

[Bastien]

Hi Stefan,

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

> The quoting behavior will be changed.

Can you quickly summarise and say what is set in stone and
what is still subject to change wrt the new quoting behavior?

I see new things like the \= in docstrings.  Will it last?

It feels like those changes are experimental: maybe we could
isolate them in a branch before merging the whole stabilised
change at once?

Thanks,

-- 
 Bastien

Re: text-quoting-style

[Paul Eggert]

Bastien wrote:
> I see new things like the \= in docstrings.  Will it last?

\= in docstrings has been supported since Emacs 16 was released in 1985.  As the 
feature has lasted three decades, even the cautious can rely on it now.

Re: text-quoting-style

[David Kastrup]

Paul Eggert <eggert@cs.ucla.edu> writes:

> Bastien wrote:
>> I see new things like the \= in docstrings.  Will it last?
>
> \= in docstrings has been supported since Emacs 16 was released in
> 1985.  As the feature has lasted three decades, even the cautious can
> rely on it now.

You have a propensity of answering questions nobody has asked.  At any
rate:

dak@lola:/usr/local/tmp/emacs$ git grep -c  '\\\\=' 'master@{now}' |awk 'BEGIN{FS=":"};{n+=$NF};END{print n}'
755
dak@lola:/usr/local/tmp/emacs$ git grep -c  '\\\\=' 'master@{1 month ago}' |awk 'BEGIN{FS=":"};{n+=$NF};END{print n}'
379

and I don't think anybody claims that every use of ` ' that should be
preserved has been caught already.

And since this affects all code including the code written in Emacs
Lisp, all external packages are affected as well.  For example, AUCTeX:

    TeX-insert-quote is an interactive compiled Lisp function in ‘tex.el’.

    (TeX-insert-quote FORCE)

    Insert the appropriate quotation marks for TeX.
    Inserts the value of ‘TeX-open-quote’ (normally ‘‘) or ‘TeX-close-quote’
    (normally ’’) depending on the context.  If ‘TeX-quote-after-quote’
    is non-nil, this insertion works only after ".
    With prefix argument FORCE, always inserts " characters.

Needless to say, this DOC string is now wrong.

Who is going to fix all of ELPA?

-- 
David Kastrup

Re: text-quoting-style

[Paul Eggert]

David Kastrup wrote:
> I don't think anybody claims that every use of ` ' that should be
> preserved has been caught already.

No, I'm sure there are more uses.  But it's not a big deal to have a transition 
period, and despite the glitches the documentation will remain quite readable 
during the transition.

Re: text-quoting-style

[David Kastrup]

Paul Eggert <eggert@cs.ucla.edu> writes:

> David Kastrup wrote:
>> I don't think anybody claims that every use of ` ' that should be
>> preserved has been caught already.
>
> No, I'm sure there are more uses.  But it's not a big deal to have a
> transition period, and despite the glitches the documentation will
> remain quite readable during the transition.

And yet another strawman.  The topic was not "readable" but "factually
wrong".

-- 
David Kastrup

Re: text-quoting-style

[Bastien]

Hi Paul,

Paul Eggert <eggert@cs.ucla.edu> writes:

> Bastien wrote:
>> I see new things like the \= in docstrings.  Will it last?
>
> \= in docstrings has been supported since Emacs 16 was released in
> 1985.  As the feature has lasted three decades, even the cautious can
> rely on it now.

Okay, thanks for letting me know.

My request was more general: I want to know which recent change about
quoting in Emacs will stay, which one is still experimental and/or
controversial?

If all current changes are set in stone, that's easier, and having a
list of those changes will help.  Or is it all documented in etc/NEWS?

If not all changes are set in stone, then I'm wondering why they do
not happen on a dedicated branch.  Making controversial changes on the
master branch feels pushy to me.

Thanks,

-- 
 Bastien

Re: text-quoting-style

[Paul Eggert]

Bastien wrote:

> If all current changes are set in stone, that's easier, and having a
> list of those changes will help.  Or is it all documented in etc/NEWS?

It is all documented in etc/NEWS, yes.  It's also documented in the manual and 
in doc strings.  I'm not a fan of installing undocumented code.

As for "set in stone", well, nothing in Emacs is set in stone. :-)

> If not all changes are set in stone, then I'm wondering why they do
> not happen on a dedicated branch.  Making controversial changes on the
> master branch feels pushy to me.

The changes did not become controversial until after they were installed in 
master.  They weren't installed in master until after being proposed and 
discussed.  Sometimes that happens.

Re: text-quoting-style

[Dmitry Gutov]

On 09/02/2015 06:17 PM, Paul Eggert wrote:

> The changes did not become controversial until after they were installed
> in master.  They weren't installed in master until after being proposed
> and discussed.  Sometimes that happens.

They were brought up and people wrote words about them, but they never 
became a consensus. That's not what I'd call "discussed".

And specific concerns were voiced right away, even before the changes 
had been installed.

Re: text-quoting-style

[Bastien Guerry]

Paul Eggert <eggert@cs.ucla.edu> writes:

> It is all documented in etc/NEWS, yes.  It's also documented in the
> manual and in doc strings.  I'm not a fan of installing undocumented
> code.

OK, thanks.

> The changes did not become controversial until after they were
> installed in master.  They weren't installed in master until after
> being proposed and discussed.  Sometimes that happens.

Given the amount of controversy and/or misunderstanding, it's obvious
that the changes should have been more carefully discussed and tested
on a dedicated branch before.  That's life.

-- 
 Bastien

Re: text-quoting-style

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > And text-quoting-style is no solution for you either, since Emacs's
  > source code and many other documents will still come with those pesky
  > thingies.

There should not be curly quotes in Emacs source code on a regular
basis.  They might appear the definition of 'format-message' and/or
'prettify', but those are very special cases.

Are there currently curly quotes in other places than that?

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: text-quoting-style

[Alan Mackenzie]

Hello, Richard.

On Tue, Sep 01, 2015 at 06:55:42PM -0400, Richard Stallman wrote:
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]

>   > And text-quoting-style is no solution for you either, since Emacs's
>   > source code and many other documents will still come with those pesky
>   > thingies.

> There should not be curly quotes in Emacs source code on a regular
> basis.  They might appear the definition of 'format-message' and/or
> 'prettify', but those are very special cases.

> Are there currently curly quotes in other places than that?

$ grep '?' *.[ch]
, where ? is the left curly quote (entered by holding down <AlgGr>
whilst typing 2, 0, 1, 8 on the numeric keypad), in our src directory
shows 20 matches.  Most, possibly all, of these seem to be to do with
the manipulation of the curly quotes themselves.

$ find . -name '*.el' | xargs grep '?' | wc -l
show 985 matches for left curly quote in the lisp source files.  I find
this concerning.

Curly quotes aren't on any standard keyboard layout.  The general notion
of how you're meant to edit files containing them is to enable a minor
mode whereby typing ` or ' sometimes does self-insert-command, other
times inserts a curly quote.  isearch is being modified so that
(?sometimes) entering ` or ' in the search string will match the
curlies.

I don't find these workarounds at all attractive.

> -- 
> Dr Richard Stallman
> President, Free Software Foundation (gnu.org, fsf.org)
> Internet Hall-of-Famer (internethalloffame.org)
> Skype: No way! See stallman.org/skype.html.

Re: text-quoting-style

[N. Jackson]

At 14:25 -0300 on Friday 2015-08-28, Alan Mackenzie wrote:

> Unless I've missed or misunderstood some development in the last two
> months or so, the need for `text-quoting-style' still exists: if we're
> going to be doing clever things to make ` and ' get converted, or
> appear to get converted, into curly quotes, then a user must be able
> to enable/disable this conversion.

+1

Re: text-quoting-style

[Stefan Monnier]

>> Unless I've missed or misunderstood some development in the last two
>> months or so, the need for `text-quoting-style' still exists: if we're
>> going to be doing clever things to make ` and ' get converted, or
>> appear to get converted, into curly quotes, then a user must be able
>> to enable/disable this conversion.
> +1

Same as for Alan: without an explicit description of the concrete
problems you face with curly quotes, this "+1" is meaningless.


        Stefan

Re: text-quoting-style

[N. Jackson]

At 14:31 -0300 on Monday 2015-08-31, Stefan Monnier wrote:

> Same as for Alan: without an explicit description of the concrete
> problems you face with curly quotes, this "+1" is meaningless.

Sorry for the delay in posting this response, it's taken me two days to
gather my thoughts enough to come up with something remotely coherent;
there are several different issues here.

It seems clear, in principle and without need for concrete examples, that
if Emacs is going to cleverly display by default something different on
the screen than is in a file or buffer there needs to be an easy way to
turn that cleverness off and see what is actually in the file or buffer,
for those occasions when the user decides that the cleverness is not
helpful.

I see the issues under discussion (or that maybe should be under
discussion) are:

1. symbol markup
1.1. internal representation
1.2. entry in Emacs
1.3. display in Emacs
1.3.1. in text modes
1.3.2. in programming modes
1.3.3. in help
1.3.4. in info
1.4. entry and display outside Emacs

2. English quoting (single and double)
2.1. internal representation
2.2. entry in Emacs
2.3. display in Emacs
2.3.1. in text modes
2.3.2. in programming modes
2.3.3. in help
2.3.4. in info
2.4. entry and display outside Emacs

Some participants in this discussion have confabulated symbol markup and
English quoting. Personally I think this is a mistake, and sacrifices
too much for a questionable gain in simplicity. It certainly has
confused the discussion over the past few months.

For symbol markup, I think that an internal representation in the
traditional form with a grave accent and a single straight quote is
perfectly fine. Entry is easy and there are no issues for display on
legacy systems. For display on graphical displays, in help and in info
it might be nice, at the user's option, to use a proportional font,
curly quotes for English quoting, and a typewriter font for markedup
symbols. There could also be an option (although I think it would be
perverse) in text and programming modes to display symbol markup in some
way different from the internal representation of grave accent /
straight quote, such as curly quotes or bold text. Such options ought to
be able to be turned on and off by the user on a per buffer / per mode
basis.

For English quoting, I think it's great that Emacs is adding (more)
support for curly quotes. The internal representation needs to be
(obviously, I think) that each type of quote is represented by itself.
For display, each type of quote could be displayed as itself, but it
would be nice to have an option to display curly quotes as straight
quotes. (This is because in normal use I have Emacs use a monospaced
font as small as I can make it and still see, despite a slight
astigmatism, all the symbols I use (especially distinguishing `,' from
`.' and `;' from `:' and `{' from `[' from `('). At these font sizes
(with every monospaced font I've tried) curly quotes are almost
impossible to see and are generally indistinguishable from motes of dust
on my monitor. In my opinion, straight quotes just work better with a
monospaced font.) Meanwhile I can imagine that others might want the
option to display straight quotes as curly ones. However, we'd want to
be able to turn these features on and off on a per buffer / per mode
basis, because sometimes one wants to see exactly what's there, without
any magic. A smart quote feature that converts straight quotes to
appropriate left and right curly quotes as one types is great -- but
obviously it needs to be something one can turn on and off. (And, for
completeness, it would also be nice to have a feature to convert all the
quotes in a buffer to straight quotes or to appropriate left and right
curly quotes.)

I realise that much of this already exists or is in the process of
being implemented; I'm writing this to emphasise the need to be able to
turn such features on and off, on a per buffer / per mode basis.

Re: text-quoting-style

[raman]

Thank you for bringing a high-level of clarity to this discussion ---
your message shows the value of structured thinking articulated in an
easy to follow flow -- I hope your  note helps us move to a cleaner
state from where we appear to be heading at present.
-- 

Re: text-quoting-style

[Paul Eggert]

Stefan Monnier wrote:
> At the very least it should not be a Custom var.

I agree.  It's the sort of variable likely to be useful only by users who know 
how to set variables without going through customization.  And Alan can get the 
behavior he wants without it being a custom var.  So I installed a patch along 
these lines as master commit d1729604c4ddb9a5314f3d8a1e0c76536cffad85.