Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Dmitry Gutov]

On 08/16/2015 07:01 PM, Paul Eggert wrote:
> branch: master
> commit 9ce1d38890a77e93af0d20f51c53419c097200d3
> Author: Paul Eggert <eggert@cs.ucla.edu>
> Commit: Paul Eggert <eggert@cs.ucla.edu>
>
>      Use curved quotes in core elisp diagnostics

And there we go, sneaking curved quotes everywhere in the Elisp source 
files. FTR, I hate that.

>      In the core elisp files, use curved quotes in diagnostic formats,
>      so that they follow user preference as per ‘text-quoting-style’
>      rather than being hard-coded to quote `like this'.

Why do you need that? Doesn't (format "Expand `%s'? " string) use the 
"preferred quoting style"?

And if not, why don't you change it so it does?

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Alan Mackenzie]

On Mon, Aug 17, 2015 at 12:44:28AM +0300, Dmitry Gutov wrote:
> On 08/16/2015 07:01 PM, Paul Eggert wrote:
> > branch: master
> > commit 9ce1d38890a77e93af0d20f51c53419c097200d3
> > Author: Paul Eggert <eggert@cs.ucla.edu>
> > Commit: Paul Eggert <eggert@cs.ucla.edu>
> >
> >      Use curved quotes in core elisp diagnostics

> And there we go, sneaking curved quotes everywhere in the Elisp source 
> files. FTR, I hate that.

No big surprises here, but so do I.

Paul, people have got to edit these files.  Doing this has rarely been
easy.  Polluting them globally with non-working characters is going to
make this even more difficult and much less pleasant.

> >      In the core elisp files, use curved quotes in diagnostic formats,
> >      so that they follow user preference as per ‘text-quoting-style’
> >      rather than being hard-coded to quote `like this'.

This user's preference is to be able to type quote marks in Emacs source
files directly without mincing workarounds, and to be able to search for
them likewise.

Revert this change, please.

-- 
Alan Mackenzie (Nuremberg, Germany).

RE: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Drew Adams]

> > >      Use curved quotes in core elisp diagnostics
> 
> > And there we go, sneaking curved quotes everywhere in the Elisp
> > source files. FTR, I hate that.
> 
> No big surprises here, but so do I.

And I.  (No surprise here either.)

It was obvious that this "experiment" would not be just that.

> Paul, people have got to edit these files.  Doing this has rarely
> been easy.  Polluting them globally with non-working characters
> is going to make this even more difficult and much less pleasant.

But surely no such consideration is as important as Paul's sense
that `...' is, hmm, ugly?

A small sacrifice for Paul's cosmetology.
A giant leap of somekind.
The Eagle will keep landing.

> Revert this change, please.

Good luck with that.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Dmitry Gutov wrote:
> Doesn't (format "Expand `%s'? " string) use the "preferred quoting style"?

No, as that would break usages such as (format "\\`%s\\'" (car e)).  This sort 
of thing is reasonably common, so it'd be a compatibility problem to change 
‘format’ to replace accent-grave.

Alan Mackenzie wrote:
> This user's preference is to be able to type quote marks in Emacs source
> files directly without mincing workarounds, and to be able to search for
> them likewise.

You can continue to type quote marks as before, and Emacs will behave as it 
always has.  The only change here is if that if you want quotes in diagnostics 
to come out in the user's preferred style, a "mincing workaround" of some sort 
is needed.

By the way, it's easy to search for quote marks in either style: just type C-s ` 
or C-s '.  And it's easy to type curved quote marks by using Electric Quote 
mode.  These are both new features.  If you don't want to use these new 
features, that's fine too.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Bastien Guerry]

Hi Paul,

Paul Eggert <eggert@cs.ucla.edu> writes:

> By the way, it's easy to search for quote marks in either style: just
> type C-s ` or C-s '.  And it's easy to type curved quote marks by
> using Electric Quote mode.  These are both new features.  If you don't
> want to use these new features, that's fine too.

I frequently use find-grep to find bits of code I'm interesting in.

I sometimes use something like

  find . -type f -exec grep --color -nH -e "\`.\+'" {} +

from M-x fing-grep to match variables in docstrings among various
files.

If the code is using ‘...’ quotes, how can I grep to match quoted
strings?  Also, electric-quote-mode does not work in the minibuffer,
and I just don't know how to type ‘ ... any hint here?

Thanks,

-- 
 Bastien

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Dmitry Gutov]

On 08/17/2015 06:44 AM, Paul Eggert wrote:

> No, as that would break usages such as (format "\\`%s\\'" (car e)).
> This sort of thing is reasonably common, so it'd be a compatibility
> problem to change ‘format’ to replace accent-grave.

But it's not a compatibility problem to replace a relatively-rare 
combination of characters? Just because the odds of it coming up are lower?

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Dmitry Gutov]

On 08/17/2015 06:44 AM, Paul Eggert wrote:

> You can continue to type quote marks as before, and Emacs will behave as
> it always has.  The only change here is if that if you want quotes in
> diagnostics to come out in the user's preferred style, a "mincing
> workaround" of some sort is needed.

You could use something like uLSQM", employed in the C code.

> By the way, it's easy to search for quote marks in either style: just
> type C-s ` or C-s '.  And it's easy to type curved quote marks by using
> Electric Quote mode.  These are both new features.  If you don't want to
> use these new features, that's fine too.

Enabling electric-quote-mode means *all* quotes will be translated to 
curvy ones in the source code one types from there on, automatically.

Bye-bye straight quotes.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Alan Mackenzie]

Hello, Paul.

On Sun, Aug 16, 2015 at 08:44:25PM -0700, Paul Eggert wrote:
> Dmitry Gutov wrote:
> > Doesn't (format "Expand `%s'? " string) use the "preferred quoting style"?

> No, as that would break usages such as (format "\\`%s\\'" (car e)).  This sort 
> of thing is reasonably common, so it'd be a compatibility problem to change 
> ‘format’ to replace accent-grave.

> Alan Mackenzie wrote:
> > This user's preference is to be able to type quote marks in Emacs source
> > files directly without mincing workarounds, and to be able to search for
> > them likewise.

> You can continue to type quote marks as before, and Emacs will behave as it 
> always has.

This may be true, but is completely irrelevant to the points I am making.

> The only change here is if that if you want quotes in diagnostics to
> come out in the user's preferred style, a "mincing workaround" of some
> sort is needed.

The main change here is that from now on, particularly if so-called
Electric Quote mode [*] is used, we're going to end up with a chaotic
mix of ascii quotes and curly quotes in our source code.  So are users
in their source code.

[*] "Electric Quote Mode" isn't an electric mode at all (see definition
in electric.el) - it'a a translation mode, and would, perhaps, be less
confusing if renamed "Translate Quote Mode".

These source files, stuffed full of non-working characters, are more
difficult to work with, whether with less, or with grep, or whatever,
and even inside of Emacs.

> By the way, it's easy to search for quote marks in either style: just type C-s ` 
> or C-s '.  And it's easy to type curved quote marks by using Electric Quote 
> mode.

Electric Quote Mode is a global mode.  It desperately needs to be buffer
local.  As it is, if EQM is enabled, non-working characters are going to
be accidentally inserted into buffers where they do not belong.

> These are both new features.  If you don't want to use these new
> features, that's fine too.

I don't want to be lumbered with editing source files with non-working
characters.

Paul, these changes of yours are simply the Wrong Thing.  The fact that
one "needs" such flaccid workarounds like EQM and the folding of quote
characters with quote characters in (some of) the searching code should
be taken as a hint just to stop and think hard.

I predict that there will be an open-ended series of bugs following from
this, little things that just don't quite work anymore, and their fixing
will be difficult and introduce yet more complexity and more little
things that don't quite work.

Emacs is steadily getting more complicated, and these changes are
gratuitous complexity.  I would say that Emacs's biggest danger at the
moment is disappearing up its own complexity.  We should be resisting
this process, not accelerating it.

-- 
Alan Mackenzie (Nuremberg, Germany).

RE: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Drew Adams]

> Paul, these changes of yours are simply the Wrong Thing.  The fact
> that one "needs" such flaccid workarounds like EQM and the folding of
> quote characters with quote characters in (some of) the searching code
> should be taken as a hint just to stop and think hard.

Amen.  Handwriting on wall.  Look.  See.

> I predict that there will be an open-ended series of bugs following
> from this, little things that just don't quite work anymore, and their
> fixing will be difficult and introduce yet more complexity and more
> little things that don't quite work.
> 
> Emacs is steadily getting more complicated, and these changes are
> gratuitous complexity.  I would say that Emacs's biggest danger at
> the moment is disappearing up its own complexity.  We should be
> resisting this process, not accelerating it.

Yes.  As I said a while back:

  So far, my impression is that there have been a shiPload of
  deep changes that try to work around all kinds of problems
  that have been introduced - and to work around the problems
  that those workarounds have introduced...

  I haven't seen such an invasive change in a long time (never?).
  Makes Rube Goldberg machines look Occam-elegant.  And for
  what?

For what?  Seriously.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Bastien Guerry wrote:
> I sometimes use something like
>
>    find . -type f -exec grep --color -nH -e "\`.\+'" {} +
>
> from M-x fing-grep to match variables in docstrings among various
> files.
>
> If the code is using ‘...’ quotes, how can I grep to match quoted
> strings?

You can use curved quotes in the grep pattern.  I just now tried the above 
command in the Emacs source code and got 108,000 hits, mostly false alarms.  I 
had better luck with this:

find . -name test -prune -o '(' -name '*.el' -o -name '*.c' ')' -exec \
   grep --color -nH -Ee \
   "[\`'‘][[:alpha:]][[:alnum:]]*-[[:alnum:]-]*[[:alnum:]][’']" \
   {} +

This yields only 39,000 hits, mostly not false alarms.  Although this is the 
sort of thing one would want to package up rather than type by hand, the curved 
quotes aren't the major reason for needing packaging.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Dmitry Gutov wrote:
> On 08/17/2015 06:44 AM, Paul Eggert wrote:
>
>> No, as that would break usages such as (format "\\`%s\\'" (car e)).
>> This sort of thing is reasonably common, so it'd be a compatibility
>> problem to change ‘format’ to replace accent-grave.
>
> But it's not a compatibility problem to replace a relatively-rare combination of
> characters? Just because the odds of it coming up are lower?

Yes, of course.  When we change how Emacs behaves, it's important to see how 
likely these changes break existing uses in significant ways.  If the odds are 
significantly nonzero, as would be the case if we changed the behavior of 
(format "...`..." ...), then we have a significant compatibility problem.  In 
contrast, if the odds are very low -- the latter being true for 
(substitute-command-keys "...`...")) -- then we should be OK.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Dmitry Gutov wrote:
> You could use something like uLSQM", employed in the C code.

You mean, instead of this:

    (message "Buffer ‘%s’ is read only." buf)

we do this:

    (message (concat "Buffer "uLSQM"%s"uRSQM" is read only.") buf)

where uLSQM and uRSQM are global string constants?  I did consider that, but 
rejected it as it would make Lisp code too hard to read and maintain.  uLSQM and 
uRSQM are needed in C code due to C99's limitations, but at least there it is 
reasonably rare (fewer than a hundred uses) and C is already pretty ugly so we 
can put up with it.  If we did the above sort of thing to Lisp we'd need to 
uglify thousands of uses, which would not be a good thing.

By the way I considered many other possibilities, and you're welcome to bring up 
other alternatives you're interested in.  Maybe we can come up with something 
better.  The point of this change is to fix a real problem, after all, not to 
stir up trouble.

> Enabling electric-quote-mode means *all* quotes will be translated to curvy ones in the source code one types from there on, automatically.

No, just the grave accents in strings and comments, and any matching 
apostrophes; this is a considerably lighter touch than what the above remark 
implies.  Occasionally one needs to type C-q ` when one really needs a grave 
accent inside a string, but that's pretty rare unless one is hard-coding 
grave-accent quoting which is something we should be moving away from nowadays 
anyway.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Eli Zaretskii]

This change (or maybe one of the previous ones in this series) caused
the following messed-up message to be displayed during the build:

  ./temacs --batch --load loadup bootstrap
  [...]
  Loading minibuffer...
  Loading d:/gnu/git/emacs/trunk/lisp/abbrev.el (source)...
  lisp/abbrev.el: Γארwith-wrapper-hookΓאש is an obsolete macro (as of 24.4); use a <foo>-function variable modified by `add-function'.
  Loading d:/gnu/git/emacs/trunk/lisp/simple.el (source)...
  lisp/simple.el: Γארwith-wrapper-hookΓאש is an obsolete macro (as of 24.4); use a <foo>-function variable modified by `add-function'.

This is displayed on a Windows console which is incapable of showing
curved quotes.  Its codeset is cp862.

To reproduce this, I do the following:

 . remove lisp/abbrev.elc and lisp/simple.elc
 . touch some of the src/*.c files
 . type "make" and watch the fun

All the other diagnostics I've seen are perfectly legible.

Let me know if I can help you with additional information.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Alan Mackenzie wrote:

> The main change here is that from now on, particularly if so-called
> Electric Quote mode [*] is used, we're going to end up with a chaotic
> mix of ascii quotes and curly quotes in our source code.

Although I also would prefer a simpler approach (one that consistently uses 
curved quotes), you've objected to that, necessitating a "chaotic" compromise.

> The fact that
> one "needs" such flaccid workarounds like EQM and the folding of quote
> characters with quote characters in (some of) the searching code should
> be taken as a hint just to stop and think hard.

For years Emacs has had significant problems in editing and searching and 
generating non-ASCII text.  Making Emacs better in this area will inevitably 
have teething problems, and we'll inevitably come up with worse solutions before 
coming up with better ones.  But we shouldn't just do nothing: these are real 
problems that need to be addressed.

Insisting that Emacs developers live and work in an ASCII ghetto has contributed 
to these problems, as it has led us to discount the importance of non-ASCII 
editing in the real world.  (I've been guilty of this as the next guy, by the 
way -- I'm not trying to cast aspersions on anybody in particular.)  It'll be 
helpful to break out of these old mindsets, and if regularly using a few 
non-ASCII characters in Emacs source will help us do that, then that'll be a 
good thing.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Bastien]

Paul Eggert <eggert@cs.ucla.edu> writes:

> This yields only 39,000 hits, mostly not false alarms.  Although this
> is the sort of thing one would want to package up rather than type by
> hand, the curved quotes aren't the major reason for needing
> packaging.

Okay, but how do you enter the curved quote in the minibuffer?

FWIW i use a Thinkpad X200.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Bastien wrote:
> how do you enter the curved quote in the minibuffer?
>
> FWIW i use a Thinkpad X200.

C-x 8 [ and C-x 8 ] should work on any platform.  A-[ and A-] might work on your 
Thinkpad, depending on how it's set up.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Alan Mackenzie]

Hello, Paul.

On Mon, Aug 17, 2015 at 09:53:37AM -0700, Paul Eggert wrote:
> Alan Mackenzie wrote:

> > The main change here is that from now on, particularly if so-called
> > Electric Quote mode [*] is used, we're going to end up with a chaotic
> > mix of ascii quotes and curly quotes in our source code.

> Although I also would prefer a simpler approach (one that consistently uses 
> curved quotes), you've objected to that, necessitating a "chaotic" compromise.

> > The fact that one "needs" such flaccid workarounds like EQM and the
> > folding of quote characters with quote characters in (some of) the
> > searching code should be taken as a hint just to stop and think
> > hard.

> For years Emacs has had significant problems in editing and searching and 
> generating non-ASCII text.

Just to be entirely clear, I've got nothing against "non-ASCII" text.
I've been typing and editing text with characters like £, ä, ß, Ü for
decades, and would not use a program which prevented me from doing so.

What I object to is _non-working_ characters - characters which appear
on nobody's keyboard (see Bastien's question about typing curly quotes)
and are problematic to display (See Eli's recent post, for example).  I
would object equally if you were, say, to insist that all symbol names
had to be written in Greek characters, and you converted all our source
files overnight to this convention.  With all due respect to our friends
who grok Greek.

> Making Emacs better in this area will inevitably have teething
> problems, and we'll inevitably come up with worse solutions before
> coming up with better ones.

Imposing non-working characters on Emacs hackers has no connection with
problems users may have with non-ascii (working-)characters.  They're
two distinct themes.

> But we shouldn't just do nothing: these are real problems that need to
> be addressed.

The two problems are separate.  I'm not aware of the problems with
non-ascii working-characters, probably because I use only Latin based
scripts.  If there are problems there, let them be solved!

But I dispute that using ` and ' as quoting marks, in contexts used by
hackers, is a problem.  If it is, it is less of a problem than those
caused by having non-working chracters throughout our sources.

> Insisting that Emacs developers live and work in an ASCII ghetto has contributed 
> to these problems, as it has led us to discount the importance of non-ASCII 
> editing in the real world.  (I've been guilty of this as the next guy, by the 
> way -- I'm not trying to cast aspersions on anybody in particular.)

Again, nobody's insisting this - the topic is working characters vs.
non-working characters, not ascii vs. the rest.

> It'll be helpful to break out of these old mindsets, and if regularly
> using a few non-ASCII characters in Emacs source will help us do that,
> then that'll be a good thing.

Again, nobody objects to the non-ascii characters - it's the non-working
characters which are the problem.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Bastien]

Paul Eggert <eggert@cs.ucla.edu> writes:

> Bastien wrote:
>> how do you enter the curved quote in the minibuffer?
>>
>> FWIW i use a Thinkpad X200.
>
> C-x 8 [ and C-x 8 ] should work on any platform.  A-[ and A-] might
> work on your Thinkpad, depending on how it's set up.

Okay, thanks.

-- 
 Bastien

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Dmitry Gutov]

On 08/17/2015 07:42 PM, Paul Eggert wrote:

> You mean, instead of this:
>
>     (message "Buffer ‘%s’ is read only." buf)
>
> we do this:
>
>     (message (concat "Buffer "uLSQM"%s"uRSQM" is read only.") buf)

Alternatively, `format' could grow format sequences that output quotes. 
The main options are: %lq and %rq, or %qs (like GCC uses). In the latter 
case, it would be used like this:

(message "Buffer %qs is read only." buf)

or

(format "See also %qs." 'car-safe)

Looks rather clean to me, but requires explicit intent on the part of 
the caller. Which seems like another benefit to me.

> By the way I considered many other possibilities, and you're welcome to
> bring up other alternatives you're interested in.  Maybe we can come up
> with something better.  The point of this change is to fix a real
> problem, after all, not to stir up trouble.

Honestly, the problem seems a lot less real to me than it does to you 
(like 1 out of 5). Failing to address it in a satisfactory manner, 
means, to me, that the work should be reverted, not blaze ahead.

But, of course, the sunk costs are considerable now, and one would 
resist backpedaling just because of that.

> No, just the grave accents in strings and comments, and any matching
> apostrophes; this is a considerably lighter touch than what the above
> remark implies.

This might seem like a fine distinction to you (and it does indicate 
smaller odds of typing something wrong in the actual source code), but 
that's exactly what I meant.

The discussion started with a certain fraction of developers disliking 
the idea of using curly quotes as markup, and that's where 
electric-quotes-mode leads.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Dmitry Gutov]

On 08/17/2015 07:53 PM, Paul Eggert wrote:

> Although I also would prefer a simpler approach (one that consistently
> uses curved quotes), you've objected to that, necessitating a "chaotic"
> compromise.

"Necessitating", here, is a matter of opinion.

> Insisting that Emacs developers live and work in an ASCII ghetto has
> contributed to these problems, as it has led us to discount the
> importance of non-ASCII editing in the real world.  (I've been guilty of
> this as the next guy, by the way -- I'm not trying to cast aspersions on
> anybody in particular.)  It'll be helpful to break out of these old
> mindsets, and if regularly using a few non-ASCII characters in Emacs
> source will help us do that, then that'll be a good thing.

The image that comes to mind, is of an odd kid who's late to a party (to 
which he's been invited for the first time), who wants to fit in, and to 
that end, drinks too much. It makes me sad.

Unicode characters are good, we should use them in text, but not in the 
basic syntax of the language or its environment.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Dmitry Gutov wrote:
> (message "Buffer %qs is read only." buf)

I could easily implement that and turn existing uses of "‘%s’" to "%qs", if you 
think it'd help.  It's a bit more complicated and it would not address all the 
uses of curved quotes in diagnostics, but it would address many of them.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Dmitry Gutov]

On 08/17/2015 09:38 PM, Paul Eggert wrote:

>> (message "Buffer %qs is read only." buf)
>
> I could easily implement that and turn existing uses of "‘%s’" to "%qs",
> if you think it'd help.

I think so.

> It's a bit more complicated and it would not
> address all the uses of curved quotes in diagnostics, but it would
> address many of them.

The other uses, which currently don't employ a formatting sequence, can 
be changed to use %qs as well. It'll just be less of a mechanical 
conversion.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Eli Zaretskii wrote:
> This is displayed on a Windows console which is incapable of showing
> curved quotes.  Its codeset is cp862.

Thanks for reporting that.  I reproduced the problem on my Fedora box by running 
with --batch in the C locale, and fixed it in master commit 
7f2b98d09d113e0f9b1fffb0524622adfafe3ac4.  Please give it a try.  If the fix 
doesn't work for you, could you please investigate why the new using_utf8 
function returns true in your environment?  It should return false.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Eli Zaretskii]

> Date: Mon, 17 Aug 2015 12:06:09 -0700
> From: Paul Eggert <eggert@cs.ucla.edu>
> CC: emacs-devel@gnu.org
> 
> Eli Zaretskii wrote:
> > This is displayed on a Windows console which is incapable of showing
> > curved quotes.  Its codeset is cp862.
> 
> Thanks for reporting that.  I reproduced the problem on my Fedora box by running 
> with --batch in the C locale, and fixed it in master commit 
> 7f2b98d09d113e0f9b1fffb0524622adfafe3ac4.  Please give it a try.

That fixed the problem, thanks.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Artur Malabarba]

[-- Attachment #1: Type: text/plain, Size: 314 bytes --]

On Aug 17, 2015 7:50 PM, "Dmitry Gutov" <dgutov@yandex.ru> wrote:
>
> On 08/17/2015 09:38 PM, Paul Eggert wrote:
>
>>> (message "Buffer %qs is read only." buf)
>>
>>
>> I could easily implement that and turn existing uses of "‘%s’" to "%qs",
>> if you think it'd help.
>
>
> I think so.

Me too.

[-- Attachment #2: Type: text/html, Size: 536 bytes --]

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 1460 bytes --]

Dmitry Gutov wrote:

>> It's a bit more complicated and it would not
>> address all the uses of curved quotes in diagnostics, but it would
>> address many of them.
>
> The other uses, which currently don't employ a formatting sequence, can be
> changed to use %qs as well. It'll just be less of a mechanical conversion.

Do you mean replacing this sort of thing:

   (message "Press ‘?’ or ‘h’ for help, ‘q’ to quit")

with this?

   (message "Press %qs or %qs for help, %qs to quit" "?" "h" "q")

If so, this doesn't sound like a good idea, as it would make the code harder to 
read.  Also, it wouldn't suffice for code like this:

     (insert (symbol-name type)
             (format " is a type (of kind ‘"))
     (help-insert-xref-button (symbol-name metatype)
                              'cl-help-type metatype)
     (insert (format "’)"))

which formats the matching quotes separately.  Of course in general one could 
rewrite even the latter example to use %qs (if only to grab the quote characters 
out of the result string and reuse them individually!) but the rewritten version 
would be significantly harder to read and maintain.

As we need to support formatting individual curved quotes anyway, there is an 
argument for keeping it simple and omitting the attached patch for paired 
quotes.  With all this in mind, do you still think the complexity of the 
attached draft patch is a good idea?

[-- Attachment #2: 0001-New-q-flag-for-format.txt --]
[-- Type: text/plain, Size: 7673 bytes --]

From 39d03d94f95e3ccf148f36ca3bfc1ee85f4dea20 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@Penguin.CS.UCLA.EDU>
Date: Mon, 17 Aug 2015 16:35:06 -0700
Subject: [PATCH] =?UTF-8?q?New=20q=20flag=20for=20=E2=80=98format=E2=80=99?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

* doc/lispref/processes.texi (Sentinels):
Don't hardwire grave quoting style in example.
* doc/lispref/strings.texi (Formatting Strings):
* etc/NEWS:
Document new q flag.
* src/editfns.c (Fformat): Implement it.
---
 doc/lispref/processes.texi |  4 +--
 doc/lispref/strings.texi   |  9 ++++--
 etc/NEWS                   | 13 +++++---
 src/editfns.c              | 78 +++++++++++++++++++++++++++++++++++++++-------
 4 files changed, 84 insertions(+), 20 deletions(-)

diff --git a/doc/lispref/processes.texi b/doc/lispref/processes.texi
index 2bc6a18..98b3dfb 100644
--- a/doc/lispref/processes.texi
+++ b/doc/lispref/processes.texi
@@ -1720,13 +1720,13 @@ sentinel, the eventual call to the sentinel will use the new one.
 @group
 (defun msg-me (process event)
    (princ
-     (format "Process: %s had the event `%s'" process event)))
+     (format "Process: %s had the event ‘%s’" process event)))
 (set-process-sentinel (get-process "shell") 'msg-me)
      @result{} msg-me
 @end group
 @group
 (kill-process (get-process "shell"))
-     @print{} Process: #<process shell> had the event `killed'
+     @print{} Process: #<process shell> had the event ‘killed’
      @result{} #<process shell>
 @end group
 @end smallexample
diff --git a/doc/lispref/strings.texi b/doc/lispref/strings.texi
index 3093338..8de1473 100644
--- a/doc/lispref/strings.texi
+++ b/doc/lispref/strings.texi
@@ -936,7 +936,7 @@ where curved single quotes stand for themselves:
 (format "The name of this buffer is ‘%s’." (buffer-name))
      @result{} "The name of this buffer is ‘strings.texi’."
 
-(format "The buffer object prints as ‘%s’." (current-buffer))
+(format "The buffer object prints as %qs." (current-buffer))
      @result{} "The buffer object prints as ‘strings.texi’."
 
 (format "The octal value of %d is %o,
@@ -1011,13 +1011,16 @@ specifier, if any, to be inserted on the right rather than the left.
 If both @samp{-} and @samp{0} are present, the @samp{0} flag is
 ignored.
 
+  The flag @samp{q} quotes the printed representation as per the
+variable @samp{text-quoting-style} described below.
+
 @example
 @group
 (format "%06d is padded on the left with zeros" 123)
      @result{} "000123 is padded on the left with zeros"
 
-(format "%-6d is padded on the right" 123)
-     @result{} "123    is padded on the right"
+(format "%q-6d is padded on the right" 123)
+     @result{} "‘123   ’ is padded on the right"
 
 (format "The word ‘%-7s’ actually has %d letters in it."
         "foo" (length "foo"))
diff --git a/etc/NEWS b/etc/NEWS
index ec3d25c..6058f22 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -912,12 +912,17 @@ create a string, and may return its first argument if the argument
 already has the correct value.
 
 +++
+** New ‘format’ flag ‘q’
+The new ‘q’ flag causes ‘format’ to quote the output representation as
+per the value of ‘text quoting-style’.  E.g., (format "%qs failed"
+"foo") might return "‘foo’ failed".
+
++++
 ** substitute-command-keys now replaces quotes.
 That is, it converts documentation strings' quoting style as per the
-value of ‘text-quoting-style’ as described above.  Doc strings in
-source code can use either curved quotes or grave accent and
-apostrophe.  As before, isolated apostrophes and characters preceded
-by \= are output as-is.
+value of ‘text-quoting-style’.  Doc strings in source code can use
+either curved quotes or grave accent and apostrophe.  As before,
+isolated apostrophes and characters preceded by \= are output as-is.
 
 +++
 ** The character classes [:alpha:] and [:alnum:] in regular expressions
diff --git a/src/editfns.c b/src/editfns.c
index ed57d8a..0e1b0c8 100644
--- a/src/editfns.c
+++ b/src/editfns.c
@@ -3822,7 +3822,7 @@ specifiers, as follows:
 
   %<flags><width><precision>character
 
-where flags is [+ #-0]+, width is [0-9]+, and precision is .[0-9]+
+where flags is [+ #-0q]+, width is [0-9]+, and precision is .[0-9]+
 
 The + flag character inserts a + before any positive number, while a
 space inserts a space before any positive number; these flags only
@@ -3835,6 +3835,9 @@ The # flag means to use an alternate display form for %o, %x, %X, %e,
 for %e, %f, and %g, it causes a decimal point to be included even if
 the precision is zero.
 
+The q flag means to quote the printed representation as per
+‘text-quoting-style’.  E.g., "%qs" is equivalent to "‘%s’".
+
 The width specifier supplies a lower limit for the length of the
 printed representation.  The padding, if any, normally goes on the
 left, but it goes on the right if the - flag is present.  The padding
@@ -3973,11 +3976,12 @@ usage: (format STRING &rest OBJECTS)  */)
 	     digits to print after the '.' for floats, or the max.
 	     number of chars to print from a string.  */
 
-	  bool minus_flag = 0;
-	  bool  plus_flag = 0;
-	  bool space_flag = 0;
-	  bool sharp_flag = 0;
-	  bool  zero_flag = 0;
+	  bool minus_flag = false;
+	  bool  plus_flag = false;
+	  bool space_flag = false;
+	  bool sharp_flag = false;
+	  bool  zero_flag = false;
+	  bool quote_flag = false;
 	  ptrdiff_t field_width;
 	  bool precision_given;
 	  uintmax_t precision = UINTMAX_MAX;
@@ -3988,11 +3992,12 @@ usage: (format STRING &rest OBJECTS)  */)
 	    {
 	      switch (*++format)
 		{
-		case '-': minus_flag = 1; continue;
-		case '+':  plus_flag = 1; continue;
-		case ' ': space_flag = 1; continue;
-		case '#': sharp_flag = 1; continue;
-		case '0':  zero_flag = 1; continue;
+		case '-': minus_flag = true; continue;
+		case '+':  plus_flag = true; continue;
+		case ' ': space_flag = true; continue;
+		case '#': sharp_flag = true; continue;
+		case '0':  zero_flag = true; continue;
+		case 'q': quote_flag = true; continue;
 		}
 	      break;
 	    }
@@ -4121,6 +4126,20 @@ usage: (format STRING &rest OBJECTS)  */)
 	      if (convbytes && multibyte && ! STRING_MULTIBYTE (args[n]))
 		convbytes = count_size_as_multibyte (SDATA (args[n]), nbytes);
 
+	      if (quote_flag)
+		{
+		  convbytes += 2;
+		  if (quoting_style == CURVE_QUOTING_STYLE)
+		    {
+		      if (!multibyte)
+			{
+			  multibyte = true;
+			  goto retry;
+			}
+		      convbytes += 4;
+		    }
+		}
+
 	      padding = width < field_width ? field_width - width : 0;
 
 	      if (max_bufsize - padding <= convbytes)
@@ -4128,6 +4147,27 @@ usage: (format STRING &rest OBJECTS)  */)
 	      convbytes += padding;
 	      if (convbytes <= buf + bufsize - p)
 		{
+
+		  if (quote_flag)
+		    {
+		      switch (quoting_style)
+			{
+			case CURVE_QUOTING_STYLE:
+			  memcpy (p, uLSQM, 3);
+			  p += 3;
+			  break;
+
+			case GRAVE_QUOTING_STYLE:
+			  *p++ = '`';
+			  break;
+
+			case STRAIGHT_QUOTING_STYLE:
+			  *p++ = '\'';
+			  break;
+			}
+		      nchars++;
+		    }
+
 		  if (! minus_flag)
 		    {
 		      memset (p, ' ', padding);
@@ -4157,6 +4197,22 @@ usage: (format STRING &rest OBJECTS)  */)
 		      nchars += padding;
 		    }
 
+		  if (quote_flag)
+		    {
+		      switch (quoting_style)
+			{
+			case CURVE_QUOTING_STYLE:
+			  memcpy (p, uRSQM, 3);
+			  p += 3;
+			  break;
+
+			default:
+			  *p++ = '\'';
+			  break;
+			}
+		      nchars++;
+		    }
+
 		  /* If this argument has text properties, record where
 		     in the result string it appears.  */
 		  if (string_intervals (args[n]))
-- 
2.1.0

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Alan Mackenzie wrote:
> What I object to is_non-working_  characters - characters which appear
> on nobody's keyboard (see Bastien's question about typing curly quotes)
> and are problematic to display (See Eli's recent post, for example).

So you object to having form-feeds in Elisp source code as well?  After all, 
form-feeds are "non-working" by your definition.  :-)

As for the philosophical issue, I'm afraid we'll just have to disagree.  In my 
experience, hackers prefer to scratch the itches they feel personally. 
Discouraging non-ASCII characters even in relatively innocuous contexts like doc 
strings and diagnostics is connected to putting off the task of making it easier 
to edit text with these characters.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Now that ordinary quotes in doc strings can appear as curly quotes
on terminals which support them, what benefit is there in putting
any real curly quotes into those doc strings?

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > The main change here is that from now on, particularly if so-called
  > > Electric Quote mode [*] is used, we're going to end up with a chaotic
  > > mix of ascii quotes and curly quotes in our source code.

  > Although I also would prefer a simpler approach (one that consistently uses 
  > curved quotes), you've objected to that, necessitating a "chaotic" compromise.

There is a much simpler solution: don't use curly quotes in doc
strings in source code.

  > For years Emacs has had significant problems in editing and searching and 
  > generating non-ASCII text.  Making Emacs better in this area will inevitably 
  > have teething problems, and we'll inevitably come up with worse solutions before 
  > coming up with better ones.

These are totally separate areas.  By all means work on improving them,
but that is no reason to have curly quotes in doc strings in source code.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Stephen J. Turnbull]

Dmitry Gutov writes:

 > Unicode characters are good, we should use them in text, but not in the 
 > basic syntax of the language or its environment.

Oh, so you want a computer language where characters are used only in
strings?  Good trick, that.

Seriously, now that we do have Unicode, and good implementations of it
(although Emacs's isn't complete yet, it's certainly usable), there's
really no excuse for _a priori_ restricting the character set used in
a computer language.  Yes, discipline is necessary: the *size* of the
character set (aside from identifier constituents) should not be
expanded without good reason.  But which characters are used shouldn't
be decided on the basis of historically limited charsets.  They should
be chosen because they are appropriate to their syntactic roles.

Backward compatibility is important.  The old-timers have a point --
the ASCII workarounds we've used for decades still work, and adding
new synonyms or changing the syntax to substitute more accurate
versions is costly to experienced users and developers.  I personally
agree with Paul -- the appropriate place to experiment with this kind
of thing is with string conventions that don't change the meaning of
Lisp programs per se, although they do affect parsing of output and
editing Lisp programs.  It's all about eating your own dogfood.  But
although I like these changes, they are hardly a no-brainer.

On the other hand, not liking input methods?  That's not admissible:
Emacs is the world's biggest, most complex input method, and that is
its primary mission.  If you can handle Emacs, you can learn a couple
dozen additional keystroke combinations to input new syntactically
significant characters (and surely the extended repertoire will
include only a few such for many years -- "a couple dozen" is a
generous concession to reactionary fears).

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Richard Stallman wrote:
> what benefit is there in putting
> any real curly quotes into those doc strings?

Format strings are easier to read and use, particularly by novices, if 
characters typically stand for themselves.  For example, typical Emacs users who 
type ‘C-h f length RET’ will see text with curved quotes like this in their 
*Help* buffers:

   ... To get the number of bytes, use ‘string-bytes’. ...

Users can cut this text from *Help* and paste it directly into their source 
code's doc strings; this is simpler than having to change those quotation marks 
to be grave accent and apostrophe.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Bastien]

Paul Eggert <eggert@cs.ucla.edu> writes:

> Format strings are easier to read and use, particularly by novices, if
> characters typically stand for themselves.

Did we ever receive a complaint from a novice about `...' readability?

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Alan Mackenzie]

Hello again, Paul.

On Mon, Aug 17, 2015 at 07:55:34PM -0700, Paul Eggert wrote:
> Alan Mackenzie wrote:
> > What I object to is_non-working_  characters - characters which appear
> > on nobody's keyboard (see Bastien's question about typing curly quotes)
> > and are problematic to display (See Eli's recent post, for example).

> As for the philosophical issue, I'm afraid we'll just have to disagree.

Sorry, you've lost me.  Which philosophical issue would that be?

> In my experience, hackers prefer to scratch the itches they feel
> personally.  Discouraging non-ASCII characters even in relatively
> innocuous contexts like doc strings and diagnostics is connected to
> putting off the task of making it easier to edit text with these
> characters.

Again, you're conflating the two different issues of non-ascii characters
and non-working characters.  These issues are separate, and don't
influence eachother.  Imposing the non-working characters left and right
curly quotes on Emacs hackers will surely make no difference as to how
easy or difficult it is to edit Russian, or Greek, or Japanese or Korean
in Emacs.  Or am I missing something?

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Dmitry Gutov]

On 08/18/2015 06:55 AM, Stephen J. Turnbull wrote:

> Oh, so you want a computer language where characters are used only in
> strings?  Good trick, that.

Ha-ha.

> Seriously, now that we do have Unicode, and good implementations of it
> (although Emacs's isn't complete yet, it's certainly usable), there's
> really no excuse for _a priori_ restricting the character set used in
> a computer language.

In theory.

> Yes, discipline is necessary: the *size* of the
> character set (aside from identifier constituents) should not be
> expanded without good reason.  But which characters are used shouldn't
> be decided on the basis of historically limited charsets.  They should
> be chosen because they are appropriate to their syntactic roles.

Historically limited or not, my keyboard, in English layout, only 
contains a given set of characters. And those are the ones we're 
comfortable typing.

> On the other hand, not liking input methods?  That's not admissible:
> Emacs is the world's biggest, most complex input method, and that is
> its primary mission.  If you can handle Emacs, you can learn a couple
> dozen additional keystroke combinations to input new syntactically
> significant characters (and surely the extended repertoire will
> include only a few such for many years -- "a couple dozen" is a
> generous concession to reactionary fears).

Why would I want to handle them? Having to use input methods adds a 
certain constant overhead, motoric and mental.

That might be fine for a language like APL, where you're forced to use 
an input method almost everywhere. There, you sacrifice the ease of 
input for succinctness across the board. That won't happen for Emacs Lisp.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Dmitry Gutov]

On 08/18/2015 02:55 AM, Paul Eggert wrote:

> Do you mean replacing this sort of thing:
>
>    (message "Press ‘?’ or ‘h’ for help, ‘q’ to quit")
>
> with this?
>
>    (message "Press %qs or %qs for help, %qs to quit" "?" "h" "q")

Yes.

> If so, this doesn't sound like a good idea, as it would make the code
> harder to read.

It looks okay to me, but if you don't like it, the first option is 
available as well: two format sequences, one for opening quote, and one 
for closing. That would be more cumbersome, though.

I've taken the idea for %qs from GCC. Do you know if they handle the 
above kind of situation somehow specially?

> Also, it wouldn't suffice for code like this:
 >
>      (insert (symbol-name type)
>              (format " is a type (of kind ‘"))
>      (help-insert-xref-button (symbol-name metatype)
>                               'cl-help-type metatype)
>      (insert (format "’)"))

But this is not about diagnostic messages anymore, right? At the moment, 
IIRC, these situations are handled by substitute-command-keys (and 
there's no need to have curly quotes in the strings here).

> which formats the matching quotes separately.  Of course in general one
> could rewrite even the latter example to use %qs (if only to grab the
> quote characters out of the result string and reuse them individually!)
> but the rewritten version would be significantly harder to read and
> maintain.

I'm definitely not suggesting that.

> As we need to support formatting individual curved quotes anyway, there
> is an argument for keeping it simple and omitting the attached patch for
> paired quotes.  With all this in mind, do you still think the complexity
> of the attached draft patch is a good idea?

This patch solves the problem of "curved quotes in core elisp 
diagnostics", which you've felt neccessary to resolve with 
9ce1d38890a77e93af0d20f51c53419c097200d3, kicking off this discussion.

So yes, I think it's valuable.

And if your point is that by having this logic in `format', we won't 
need it in `substitute-command-keys', then I stand by the assertion that 
a separate, different, function should translate the quotes.

text -> (substitute-command-keys) -> text with "escaped" text prop
      -> (translate-quotes) -> text with non-escaped straight quotes 
replaces with curly ones

`format' can't serve as `translate-quotes', because, like you said, 
"\\`%s\\'" is a relatively common format string.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Óscar Fuentes]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

[snip]

> Seriously, now that we do have Unicode, and good implementations of it
> (although Emacs's isn't complete yet, it's certainly usable), there's
> really no excuse for _a priori_ restricting the character set used in
> a computer language.
[snip]

My experience says that there are plenty of *reasons* for being
extremely cautious about using anything other than ASCII in programs,
(including (doc-)strings, to a lesser extrem.)

Just see the issues faced on this topic of curved quotes, and it is a
single case of Unicode use.

I'm all for modernization of Emacs, but this change just doesn't make
sense to me.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Yuri Khan]

On Mon, Aug 17, 2015 at 11:35 PM, Alan Mackenzie <acm@muc.de> wrote:

> What I object to is _non-working_ characters - characters which appear
> on nobody's keyboard (see Bastien's question about typing curly quotes)
> and are problematic to display (See Eli's recent post, for example).

It’s these two problems which need fixed.

We need keyboards (or input methods) which offer a convenient way to
enter typographically correct quotes and dashes and other essential
punctuation, because otherwise people fall back to their
easier-to-enter ASCII substitutes.

We need terminals which are capable of displaying the whole repertoire
of Unicode, because otherwise we have to make a choice of the subset
we’d like to be able to see.

Making these two long-standing problems more visible is a good thing.

(As far as I am concerned, both are solved problems already. It’s just
that the solutions are not mainstream enough.)

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Andreas Schwab]

Yuri Khan <yuri.v.khan@gmail.com> writes:

> (As far as I am concerned, both are solved problems already. It’s just
> that the solutions are not mainstream enough.)

In X11 a whole lot of special characters are available via Compose, but
they are not easily discoverable (see
/usr/share/X11/locale/en_US.UTF-8/Compose for the full list).

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Yuri Khan]

On Tue, Aug 18, 2015 at 9:24 PM, Andreas Schwab <schwab@suse.de> wrote:
> Yuri Khan <yuri.v.khan@gmail.com> writes:
>
>> (As far as I am concerned, both are solved problems already. It’s just
>> that the solutions are not mainstream enough.)
>
> In X11 a whole lot of special characters are available via Compose, but
> they are not easily discoverable (see
> /usr/share/X11/locale/en_US.UTF-8/Compose for the full list).

Better than that: enabling the misc:typo XKB option, along with any of
the options of the lv3:* group, puts many useful characters directly
on the keyboard, by adding two more character layers (in addition to
normal and Shift layers).

There are tools (based on xkbprint) that show the resulting layout.
GNOME 2 used to make it reasonably discoverable; I don’t know about
GNOME 3.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Alan Mackenzie]

Hello, Yuri.

On Tue, Aug 18, 2015 at 09:09:51PM +0600, Yuri Khan wrote:
> On Mon, Aug 17, 2015 at 11:35 PM, Alan Mackenzie <acm@muc.de> wrote:

> > What I object to is _non-working_ characters - characters which appear
> > on nobody's keyboard (see Bastien's question about typing curly quotes)
> > and are problematic to display (See Eli's recent post, for example).

> It’s these two problems which need fixed.

> We need keyboards (or input methods) which offer a convenient way to
> enter typographically correct quotes and dashes and other essential
> punctuation, because otherwise people fall back to their
> easier-to-enter ASCII substitutes.

There are already input methods for curly quotes (C-x 8 [ and C-x 8 ], I
believe), but whether these will ever count as "convenient", I somehow
doubt.  Even typing [ and ] on a German keyboard layout (just as an
example) is somewhat less than convenient.

One solution is to enhance our own personal keyboard layouts, but that
isn't a solution for our users.  Nothing practical we can come up with
is going to be as easy as just hitting key number 41 and key number 40.

> We need terminals which are capable of displaying the whole repertoire
> of Unicode, because otherwise we have to make a choice of the subset
> we’d like to be able to see.

Such terminals probably exist.  However, they're not what "everybody" is
using.  The Linux virtual terminal, which I use, is currently limited to
256 distinct glyphs.  I've had a look at the code, with a view to
enhancing it, but it is not well maintained and easily adaptible code.
Displaying the curly quotes in it is not unproblematic.  Yesterday, Eli
Z. reported a problem on an MS-Windows terminal which couldn't display
these characters at all.

One of the outstanding features of Emacs is that it will run equally
well in "any" environment.  This property is well worth preserving.

> Making these two long-standing problems more visible is a good thing.

Visible to whom?  We definitely don't want to make them visible to our
users.  That will just make them annoyed and resentful.

> (As far as I am concerned, both are solved problems already. It’s just
> that the solutions are not mainstream enough.)

I'd be interested in hearing a bit more about what you see as the
solutions.  Usually, you don't get something for nothing, and I'd bet
that these solutions come with their own disadvantages, compared with
what "everybody" is currently using.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Alan Mackenzie wrote:
> Sorry, you've lost me.  Which philosophical issue would that be?

It's illustrated by your next comment:

> you're conflating the two different issues of non-ascii characters
> and non-working characters.

We disagree about this.  They're not orthogonal issues.  They're so closely 
related that they're almost the same issue.  Emacs currently makes it harder to 
deal with non-ASCII and/or non-working characters than it could.  Restricting 
our source code to ASCII and/or working characters has helped us put our heads 
in the sand about the problem.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Yuri Khan]

On Tue, Aug 18, 2015 at 9:48 PM, Alan Mackenzie <acm@muc.de> wrote:

> There are already input methods for curly quotes (C-x 8 [ and C-x 8 ], I
> believe), but whether these will ever count as "convenient", I somehow
> doubt.  Even typing [ and ] on a German keyboard layout (just as an
> example) is somewhat less than convenient.

Correct. Using Emacs as the input method is not feasible for users of
other applications.

My current method of entering curly quotes is comparable to entering
brackets on a German keyboard — hold down right Alt, press a single
key. I find it good enough for such infrequent characters.

>> We need terminals which are capable of displaying the whole repertoire
>> of Unicode, because otherwise we have to make a choice of the subset
>> we’d like to be able to see.
>
> Such terminals probably exist.  However, they're not what "everybody" is
> using.

“Everybody” is not using a terminal at all. “Everybody” uses a
graphical desktop. Including, in some circumstances, a terminal
emulator. Xterm, for one, does not support all of Unicode equally
well, but curly quotes are unproblematic.

> The Linux virtual terminal, which I use, is currently limited to
> 256 distinct glyphs.

Actually 512 if you sacrifice 8 of the 16 colors, and there is
possibility of replacing it with fbterm or other framebuffer-based
terminals.

> Yesterday, Eli
> Z. reported a problem on an MS-Windows terminal which couldn't display
> these characters at all.

I used to use Windows, including the Windows console, as my primary
environment. It displays most of the European part of Unicode
allright, once you configure it to use a TrueType or OpenType font.
CJK is harder (because most Han characters want to occupy two
character cells each) and RTL is harder still, but, again, curly
quotes are unproblematic.

Some applications (notably, ports of Unix utilities) have problems
displaying Unicode on the Windows console. That is a bug in those
applications.

>> (As far as I am concerned, both are solved problems already. It’s just
>> that the solutions are not mainstream enough.)
>
> I'd be interested in hearing a bit more about what you see as the
> solutions.  Usually, you don't get something for nothing, and I'd bet
> that these solutions come with their own disadvantages, compared with
> what "everybody" is currently using.

For output, the solution is a graphical environment. With TrueType,
OpenType or otherwise vector-based scalable fonts, rendered through a
facility which supports Unicode, ligatures, combining diacritics, RTL,
complex scripts, rich formatting and whatnot.


For input, my current setup involves two layouts (for English/Latin
and Russian/Cyrillic), which differ in their 1st and 2nd levels, but
have common 3rd and 4th levels, activated with right Alt with and
without Shift. This accommodates like 99.99% of my typing needs. For
the remaining cases, I resort to a character map application or to
Emacs’ insert-char.

I don’t think I am losing much for it. I am vaguely aware that having
two Alt keys is more convenient than just one but, to be frank, I also
under-use right Shift and right Ctrl. (My first computer did not have
a right Shift; that might have influenced my typing habits.)

New keyboard designs are emerging which provide more keys intended to
be pressed with thumbs. This is ideal for multi-level layouts. The
classic AT keyboard with only 0.5 to 1.5 keys per thumb needs to give
way.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Alan Mackenzie]

Hello, Paul.

On Tue, Aug 18, 2015 at 09:45:38AM -0700, Paul Eggert wrote:
> Alan Mackenzie wrote:
> > Sorry, you've lost me.  Which philosophical issue would that be?

> It's illustrated by your next comment:

> > you're conflating the two different issues of non-ascii characters
> > and non-working characters.

> We disagree about this.  They're not orthogonal issues.  They're so closely 
> related that they're almost the same issue.

Well, as I said, I edit texts with non-ascii characters frequently, and
don't experience any particular difficulty with them.  Having to type in
a decimal/hex code for a non-working character (or, even worse, having
to look up an input method for it) just stops me in my tracks.  An
example is when I reply to Óscar, "Ó" being outside my working character
set.

> Emacs currently makes it harder to deal with non-ASCII and/or
> non-working characters than it could.

Could you give an example of this (pertaining, preferably, to non-ascii
working characters)?  Thinking about it, maybe having to use Latin keys
in lots of bindings when ones keyboard's preferred layout is non-Latin
could be quite tiresome.  But I've never seen anybody complaining about
this.

> Restricting our source code to ASCII and/or working characters has
> helped us put our heads in the sand about the problem.

Whatever problem that might be, the solution surely cannot be
artificially to inflict it on ourselves.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Bastien wrote:
> Paul Eggert <eggert@cs.ucla.edu> writes:
>
>> Format strings are easier to read and use, particularly by novices, if
>> characters typically stand for themselves.
>
> Did we ever receive a complaint from a novice about `...' readability?

Most novices don't bother to write bug reports -- they don't even know how to 
write bug reports.  But yes, people occasionally gripe about the use of grave 
accent to quote, and this can hurt Emacs's reputation among people who may not 
know it better.  For example, <http://wordyenglish.com/musing/typography.html> 
(2007) says:

"the problem with the GNU is that even today, in 2007, where curly quotes have 
been widely available in word processors for over a decade (and Unicode have 
been practical and widely available for at least 5 years...), they are still 
using plain ASCII hacks. (in general, GNU and the Open Source morons have like a 
5 to 10 years lag in adopting technology, for reasons that are inadvertently 
intentional and or simply incapable)"

And here we are in 2015, with the quote problem still only partly fixed.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Andreas Schwab]

Paul Eggert <eggert@cs.ucla.edu> writes:

> And here we are in 2015, with the quote problem still only partly fixed.

In 2015, people use `...`.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Eli Zaretskii]

> From: Yuri Khan <yuri.v.khan@gmail.com>
> Date: Tue, 18 Aug 2015 23:08:27 +0600
> Cc: Paul Eggert <eggert@cs.ucla.edu>, Dmitry Gutov <dgutov@yandex.ru>,
> 	Emacs developers <emacs-devel@gnu.org>
> 
> I used to use Windows, including the Windows console, as my primary
> environment. It displays most of the European part of Unicode
> allright, once you configure it to use a TrueType or OpenType font.

Europe is but a small part of the world, and the fonts available for
that on the Windows console are ugly and hard to read.

So these problems are much more significant than what you are trying
to convince us.

> curly quotes are unproblematic.

Not true, see above.

> Some applications (notably, ports of Unix utilities) have problems
> displaying Unicode on the Windows console. That is a bug in those
> applications.

LOL.  I have yet to see a native Windows port of a significant Unix
utility that doesn't have that "bug".  About the only one I know about
is the stand-alone Info reader from the latest Texinfo 6.0 release.

And once again, when all is said and done, and the ported code handles
Unicode characters correctly when writing to the console, you are back
at the console font problem I mentioned above: anything outside the
European locales is downright impossible, and inside Europe you have
only the ugly Lucida Console font.

Let's not pretend this is easy, especially on Windows and in any other
non-UTF-8 locale.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Chad Brown]

> On 18 Aug 2015, at 11:05, Andreas Schwab <schwab@linux-m68k.org> wrote:
> 
> Paul Eggert <eggert@cs.ucla.edu> writes:
> 
>> And here we are in 2015, with the quote problem still only partly fixed.
> 
> In 2015, people use `...`.


I work more in prose now than in code, and I’ve received numerous
complaints from editors in the last ~3 years about text that uses
that. In practice, its currently hard for me to type that exact
string, because all of the programs I use that aren’t Emacs or some
of the terminal console interfaces automatically turn it into ‘…’.

I’ve mostly given up on using Emacs for long-form prose work for
this reason (which is too bad, because the editing support for prose
is still quite good, and also long-since pushed down into muscle
memory). With all this in mind, I’ve been watching these exploding
threads with a mixture of hope and dread, because the modernization
effort would be very nice for me, but I’m seeing a lot of older
people whose opinions and efforts I respect argue for "just use
old-style ascii quotes."

I hope that helps. Thanks again to Paul for trying to push Emacs forward. 
~Chad

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Alan Mackenzie wrote:

> Well, as I said, I edit texts with non-ascii characters frequently, and
> don't experience any particular difficulty with them.  Having to type in
> a decimal/hex code for a non-working character (or, even worse, having
> to look up an input method for it) just stops me in my tracks.  An
> example is when I reply to Óscar, "Ó" being outside my working character
> set.
>
>> Emacs currently makes it harder to deal with non-ASCII and/or
>> non-working characters than it could.
>
> Could you give an example of this (pertaining, preferably, to non-ascii
> working characters)?

You gave an example in your previous paragraph, where you're stopped in your 
tracks if you have to type "Ó" into Emacs.

> Whatever problem that might be, the solution surely cannot be
> artificially to inflict it on ourselves.

There's nothing artificial about using a character to represent itself in 
typical usage in a doc string or a diagnostic.  What's artificial is requiring 
users to laboriously type and read ASCII-only circumlocutions instead.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Alan Mackenzie]

Hello, Chad.

On Tue, Aug 18, 2015 at 11:32:36AM -0700, Chad Brown wrote:

> > On 18 Aug 2015, at 11:05, Andreas Schwab <schwab@linux-m68k.org> wrote:
> > 
> > Paul Eggert <eggert@cs.ucla.edu> writes:
> > 
> >> And here we are in 2015, with the quote problem still only partly fixed.
> > 
> > In 2015, people use `...`.


> I work more in prose now than in code, and I’ve received numerous
> complaints from editors in the last ~3 years about text that uses
> that. In practice, its currently hard for me to type that exact
> string, because all of the programs I use that aren’t Emacs or some
> of the terminal console interfaces automatically turn it into ‘…’.

Yet, for lots of programming languages, it is essential to be able to
type `, ', and even ....  (That's three dots followed by a full stop!)

> I’ve mostly given up on using Emacs for long-form prose work for
> this reason (which is too bad, because the editing support for prose
> is still quite good, and also long-since pushed down into muscle
> memory).

It seems what you're missing in Emacs is some sort of "literary mode"
which would make the translations you need.

When you say "still quite good", this gives the impression of a steady
diminution of quality, which hasn't yet dropped below some sort of
threshold.  Is this what you actually mean?

> With all this in mind, I’ve been watching these exploding threads with
> a mixture of hope and dread, because the modernization effort would be
> very nice for me, but I’m seeing a lot of older people whose opinions
> and efforts I respect argue for "just use old-style ascii quotes."

I think you may have misunderstood what the controversy is about.  It's
not about adding or not adding support for curly quotes and curly
ellipses in users' documents.  It's about whether or not to replace `
and ' in Emacs's source files' error messages and comments with curly
quotes, for a perceived aesthetic benefit, at the cost of making those
files more difficult to maintain, particularly in environments which
aren't X-windows.

> I hope that helps. Thanks again to Paul for trying to push Emacs forward. 

We're _all_ trying to push Emacs forward.  It's just we sometimes don't
fully agree what direction forward is.  :-)

> ~Chad

-- 
Alan Mackenzie (Nuremberg, Germany).

RE: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Drew Adams]

> I think you may have misunderstood what the controversy is about.
> It's not about adding or not adding support for curly quotes and curly
> ellipses in users' documents.  It's about whether or not to replace
> ` and ' in Emacs's source files' error messages and comments with
> curly quotes, for a perceived aesthetic benefit, at the cost of making
> those files more difficult to maintain, particularly in environments which
> aren't X-windows.

AND at the cost of confusing ordinary text quotation (which, naturally,
uses curly quotes, both double and single) with mention, within passages
of ordinary text, of things that the text wants to talk about: code
fragments such as symbols and sexps, as well as key sequences, file names,
and URLS.  IOW, the kinds of things that are often distinguished from
ordinary text by setting them off in a fixed-width font (such as Courier).

IOW, regardless of the (important) problems Alan mentions, about
difficulty of use, burden of maintenance, misfit with existing
code-oriented tools, and complication of user and core code (layers and
layers of ugly workarounds) -- i.e., even if those problems were not a
problem, it is misguided to treat setting off code etc. using ordinary
text quotation.  It introduces confusion and is information lossy.

> We're _all_ trying to push Emacs forward.  It's just we sometimes
> don't fully agree what direction forward is.  :-)

Not to mention the effect on many others who will deal with the
consequences without having voiced anything about the direction.
(And yes, I'm sure there will be many end users who will not get
beyond a first blush of "Look Ma, curly quotes now!  How pretty!")

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Alan Mackenzie]

Hello, Paul.

On Tue, Aug 18, 2015 at 12:25:18PM -0700, Paul Eggert wrote:
> Alan Mackenzie wrote:

> > Well, as I said, I edit texts with non-ascii characters frequently, and
> > don't experience any particular difficulty with them.  Having to type in
> > a decimal/hex code for a non-working character (or, even worse, having
> > to look up an input method for it) just stops me in my tracks.  An
> > example is when I reply to Óscar, "Ó" being outside my working character
> > set.

> >> Emacs currently makes it harder to deal with non-ASCII and/or
> >> non-working characters than it could.

> > Could you give an example of this (pertaining, preferably, to non-ascii
> > working characters)?

> You gave an example in your previous paragraph, where you're stopped in your 
> tracks if you have to type "Ó" into Emacs.

OK, but I can't really see the connection between this (and what Chad
Brown was unhappy about) and the replacement in our source code of ` and
' by curlies.  Nobody having to type "Ó" on a Spanish keyboard layout
would have any trouble.

> > Whatever problem that might be, the solution surely cannot be
> > artificially to inflict it on ourselves.

> There's nothing artificial about using a character to represent itself in 
> typical usage in a doc string or a diagnostic.  What's artificial is requiring 
> users to laboriously type and read ASCII-only circumlocutions instead.

There is nothing laborious about hitting key 41 or key 40.  Anybody who
finds this laborious will not be programming in lisp for very long.

There is nothing indirect about using ` and ' to "stand for themselves",
in quoting things.  I've never heard of an Emacs hacker experiencing
difficulty reading things quoted `like this'.

On the contrary, holding down <AltGr> while typing on the numeric
keypad, successively 2, 0, 1, 8 or 2, 0, 1, 9 is laborious indeed, even
assuming that these codes have been retained in memory.  For that is
what a user with a normal keyboard layout, outside of Emacs, will be
forced to do.  As we know there are workarounds inside Emacs to help
with this.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Dmitry Gutov]

On 08/18/2015 08:34 PM, Paul Eggert wrote:

> Most novices don't bother to write bug reports -- they don't even know
> how to write bug reports.

Bug reports are written by users who are at least a little experienced, 
sure, but we shouldn't assume that every such user has necessarily 
become accustomed to Emacs's quirks, and wouldn't call out this problem, 
if it were a real problem.

> But yes, people occasionally gripe about the
> use of grave accent to quote, and this can hurt Emacs's reputation among
> people who may not know it better.  For example,
> <http://wordyenglish.com/musing/typography.html> (2007) says:

I sincerely hope the whole effort wasn't kicked off by this Xah Lee's 
rant. It's pretty shallow. And the author should really "know Emacs 
better" by now.

> "the problem with the GNU is that even today, in 2007, where curly
> quotes have been widely available in word processors for over a decade
> (and Unicode have been practical and widely available for at least 5
> years...), they are still using plain ASCII hacks. (in general, GNU and
> the Open Source morons have like a 5 to 10 years lag in adopting
> technology, for reasons that are inadvertently intentional and or simply
> incapable)"

"morons"... yeah.

> And here we are in 2015, with the quote problem still only partly fixed.

One would have to define the "problem" first.

In 2015, the documentation markup languages (Markdown, Asciidoc, etc) 
support rich content (images, hyperlinks, document structure), and 
decoupling markup from presentation (usually through rendering into HTML).

Yet here we are, not talking about any big features, and instead 
discussing using unicode quotes in the markup (which none of the modern 
markup languages do), because it's "easier" if the markup and 
presentation are the same. That's a step back, if anything.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Dmitry Gutov]

On 08/18/2015 09:32 PM, Chad Brown wrote:

> because all of the programs I use that aren’t Emacs or some
> of the terminal console interfaces automatically turn it into ‘…’.

All programs? None of the popular programming text editors do that.

You might be thinking of LibreOffice Writer.

> With all this in mind, I’ve been watching these exploding
> threads with a mixture of hope and dread, because the modernization
> effort would be very nice for me, but I’m seeing a lot of older
> people whose opinions and efforts I respect argue for "just use
> old-style ascii quotes."

Since you're writing prose, the discussion should be irrelevant to you.

It's only about which quotes to use in the Emacs Lisp source files.

For inserting proper punctuation marks automatically when writing prose, 
you should check out 
http://www.emacswiki.org/emacs/TypographicalPunctuationMarks

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Alan Mackenzie wrote:
> Nobody having to type "Ó" on a Spanish keyboard layout
> would have any trouble.

Spanish keyboards typically do not have "Ó", so your restrictive definition of 
"working" would say that "Ó" is trouble even on a Spanish keyboard.

> On the contrary, holding down <AltGr> while typing on the numeric
> keypad, successively 2, 0, 1, 8 or 2, 0, 1, 9 is laborious indeed, even
> assuming that these codes have been retained in memory.  For that is
> what a user with a normal keyboard layout, outside of Emacs, will be
> forced to do.

The criterion cannot be that any text editor in any configuration should be able 
to edit Emacs source code with no trouble.  That hasn't ever been true.  All 
that's needed is that people be able to edit their source code in Emacs, with 
occasional use by other text editors, when properly configured.  All modern text 
editors can handle UTF-8 text files when properly configured, so this is not a 
problem.  I just now checked 'less' and 'vim', for example, and they work fine 
with UTF-8 curved quotes even on the Linux console if it's properly configured.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Dmitry Gutov wrote:
> Since you're writing prose, the discussion should be irrelevant to you.
>
> It's only about which quotes to use in the Emacs Lisp source files.

That remark could easily be misinterpreted by those not closely following the 
discussion.  We're not talking about ", `, and ' in Emacs Lisp syntax -- 
nobody's proposing changing that.  We're talking only about the prose that 
appears in commentary and strings of Emacs Lisp source files, and this makes 
Chad Brown's remarks about prose relevant to the discussion.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Óscar Fuentes]

Paul Eggert <eggert@cs.ucla.edu> writes:

> Alan Mackenzie wrote:
>> Nobody having to type "Ó" on a Spanish keyboard layout
>> would have any trouble.
>
> Spanish keyboards typically do not have "Ó", so your restrictive
> definition of "working" would say that "Ó" is trouble even on a
> Spanish keyboard.

Every Spaniard who ever used a typewriter knows that accented letters
are composed like this: `o. This is not a problem at all and the
approach generalizes to others cases, such as French and Catalonian,
which have more accent types than Spanish. An Spaniard would have no
problem at all typing the French è, for instance. Furthermore, an
Spaniard would have no problem using an USA keyboard with the
USA-international input method: we quickly figure out that ñ is ~n.

However, I need to learn how to write the curly quotes, and then
remember the method. This makes things harder for us occasional
contributors, not to mention beginners. I'm pretty sure that copy&paste
will become more popular with this convention.

I admit that curly quotes are nicer, but that's easily achievable: make
them appear on *Help* and other buffers that shows docstrings. Dmitry
suggests this, and his comment about modern markup languages restricting
themselves to ASCII is something to think about.

If your goal is to make Emacs more Unicode-friendly, this is not the way
to go. Identify the problematic areas, taking feedback from users like
Chad Brown, and then proceed to fix those shortcomings with those users
acting as "customers" (i.e. judges of the proposed solution.)

I admit that I'm intrigued by your plan about how this change will
initiate an evolution on Emacs input system that will make easier to
type exotic characters (defining "exotic" by "something that it is
infrequent in your daily usage.") Maybe describing the specific
user-visible improvements that this change will help to bring into
reality would buy you more support.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Bastien]

Óscar Fuentes <ofv@wanadoo.es> writes:

> I admit that curly quotes are nicer, but that's easily achievable: make
> them appear on *Help* and other buffers that shows docstrings. Dmitry
> suggests this, and his comment about modern markup languages restricting
> themselves to ASCII is something to think about.

1+

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Alan Mackenzie]

Hello, Paul.

On Tue, Aug 18, 2015 at 02:40:17PM -0700, Paul Eggert wrote:
> Alan Mackenzie wrote:
> > Nobody having to type "Ó" on a Spanish keyboard layout
> > would have any trouble.

> Spanish keyboards typically do not have "Ó", so your restrictive definition of 
> "working" would say that "Ó" is trouble even on a Spanish keyboard.

I think you know full well what I mean by "working character", and that
my definition wasn't meant to be water tight in the way you're now
trying to pick holes in.  Spanish keyboards DO have "Ó", for any
sensible value of "have".

> > On the contrary, holding down <AltGr> while typing on the numeric
> > keypad, successively 2, 0, 1, 8 or 2, 0, 1, 9 is laborious indeed, even
> > assuming that these codes have been retained in memory.  For that is
> > what a user with a normal keyboard layout, outside of Emacs, will be
> > forced to do.

> The criterion cannot be that any text editor in any configuration should be able 
> to edit Emacs source code with no trouble.  That hasn't ever been true.

I think it pretty much has been, certainly for all but a few special
files, up until the last few days.

> All that's needed is that people be able to edit their source code in
> Emacs, with occasional use by other text editors, when PROPERLY
> CONFIGURED.  All MODERN text editors can HANDLE UTF-8 text files when
> PROPERLY CONFIGURED, so this is not a problem.  I just now checked
> 'less' and 'vim', for example, and they WORK FINE with UTF-8 curved
> quotes even on the Linux console if it's PROPERLY CONFIGURED.

Paul, that paragraph is pretty much content free, without having some
good idea of what that vague bits (I've capitalised them) mean.

I think you're agreeing with me that to work with curly quotes using
these tools, you're going to be having to enter their hex codes, as
described above.  In the file versions without the curlies, this simply
didn't arise.  Putting the curly quotes into our source files have made
them more difficult to work with.  This is a Bad Thing.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Óscar Fuentes wrote:
> I admit that curly quotes are nicer, but that's easily achievable: make
> them appear on*Help*  and other buffers that shows docstrings.

That was done weeks ago.  This thread is about something else: quotes in 
diagnostics.  It's not as easy to get diagnostics right, which is why we're 
having this discussion.

> Maybe describing the specific
> user-visible improvements that this change will help to bring into
> reality would buy you more support.

I have no secret master plan for revolutionizing Emacs.  I'm just trying to make 
improvements one step at a time.  If it helps, I think diagnostics are the last 
major component of getting Emacs quoting to follow common modern practice 
(controlled by user preference of course).

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Óscar Fuentes]

Paul Eggert <eggert@cs.ucla.edu> writes:

> Óscar Fuentes wrote:
>> I admit that curly quotes are nicer, but that's easily achievable: make
>> them appear on*Help*  and other buffers that shows docstrings.
>
> That was done weeks ago.  This thread is about something else: quotes
> in diagnostics.  It's not as easy to get diagnostics right, which is
> why we're having this discussion.

I thought that, in this subthread, we were discussing the introduction
of curly quotes in source code. Sorry for the confusion.

>> Maybe describing the specific
>> user-visible improvements that this change will help to bring into
>> reality would buy you more support.
>
> I have no secret master plan for revolutionizing Emacs.  I'm just
> trying to make improvements one step at a time.  If it helps, I think
> diagnostics are the last major component of getting Emacs quoting to
> follow common modern practice (controlled by user preference of
> course).

There are some people (me included) that doubt that using hard-to-type
chars in source code is an improvement, quite the opposite. I've yet to
see a justification for this change. It would be great to have an
explanation (as specific as possible) about how it will help to improve
Emacs.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Óscar Fuentes]

Óscar Fuentes <ofv@wanadoo.es> writes:

> Paul Eggert <eggert@cs.ucla.edu> writes:
>
>> Óscar Fuentes wrote:
>>> I admit that curly quotes are nicer, but that's easily achievable: make
>>> them appear on*Help*  and other buffers that shows docstrings.
>>
>> That was done weeks ago.  This thread is about something else: quotes
>> in diagnostics.  It's not as easy to get diagnostics right, which is
>> why we're having this discussion.
>
> I thought that, in this subthread, we were discussing the introduction
> of curly quotes in source code. Sorry for the confusion.

Oh, but the diagnostics are part of the source code, of course. I was
not so confused after all.

I know that docstrings are displayed with curly quotes. My point (and I
think Dmitry's) is that the change should end there. Maybe *transform*
the diagnostics too, but that's no so important, because diagnostics are
not intended to be part of regular reading by users.

[snip]

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Dmitry Gutov]

On 08/19/2015 12:43 AM, Paul Eggert wrote:
> We're talking only
> about the prose that appears in commentary and strings of Emacs Lisp
> source files, and this makes Chad Brown's remarks about prose relevant
> to the discussion.

Not at all. I'm pretty sure the "numerous complaints from editors" Chad 
mentioned were not about contents of docstrings in Elisp Chad wrote.

Whatever choice is made WRT to docstring syntax, it should have to 
bearing on people's ability to write prose (books, articles, blog posts) 
and typography used therein.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

I use non-ASCII characters every day while editing text in French and
Spanish.  I also sometimes need to enter names in other languages,
such as Hungarian and Turkish, which is why I have asked for better
support for occasionally inputting unusual (for me) non-ASCII
characters not supported by my input method.

For this reason, and on general principles, I am enthusiastically in
support of improving all the aspects of Emacs support for non-ASCII
characters.

But that is no reason to put non-ASCII characters into the conventions
for Emacs Lisp source code.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Tex mode automatically turns " into `` or '' according to the context.
Couldn't we easily have a mode for use in Text mode, that turns ' into
‘ or ’ according to the context?  (Maybe we already do.)

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > what benefit is there in putting
  > > any real curly quotes into those doc strings?

  > Format strings are easier to read and use, particularly by novices, if 
  > characters typically stand for themselves.

The issue at hand is doc strings in the source code, not format strings.
Those are a different issue.

  >    ... To get the number of bytes, use ‘string-bytes’. ...

  > Users can cut this text from *Help* and paste it directly into their source 
  > code's doc strings; this is simpler than having to change those quotation marks

It's a bad practice to copy text from a help buffer into
a doc string in the source code.  There are several constructs
used in source code doc strings that get converted into something
different in a help buffer.  So if you want to copy one doc string
into another, you should copy from source code.

I doubt that case arises very often, though.  If the user's new
function is so similar to an existing function that he wants to copy
from its doc string, he probably wrote his function by modifying the
code of that function, and probably already has its doc string.


-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Richard Stallman wrote:
> Tex mode automatically turns " into `` or '' according to the context.
> Couldn't we easily have a mode for use in Text mode, that turns ' into
> ‘ or ’ according to the context?  (Maybe we already do.)

Yes we do, in Emacs master.  It's Electric Quote mode.  You can try it out by 
putting this into your ~/.emacs file:

(if (fboundp 'electric-quote-mode)
     (electric-quote-mode))

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Alan Mackenzie wrote:

> I think you know full well what I mean by "working character",

No, actually, I don't.  From my point of view "working character" is a slippery 
notion that mutates when we try to pin it down.  Perhaps we should stick with 
"non-ASCII character"; that's clear.

> Spanish keyboards DO have "Ó", for any sensible value of "have".

Only if properly configured.  If you use these keyboards with the wrong 
settings, Compose ' O won't work, and there's no "Ó" key on the keyboard so 
you'll be stuck.  In these respects "Ó", "‘" and "’" are all in the same 
category on a Spanish keyboard.  And this is OK.

>> The criterion cannot be that any text editor in any configuration should be able
>> to edit Emacs source code with no trouble.  That hasn't ever been true.
>
> I think it pretty much has been, certainly for all but a few special
> files, up until the last few days.

No, it hasn't been true *at all*.  It's quite common for Japanese keyboards to 
lack a ‘\’ key, for example.  And this is OK too.

> I think you're agreeing with me that to work with curly quotes using
> these tools, you're going to be having to enter their hex codes

Not at all.  That would be silly.  I never use hex codes to enter these characters.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Yuri Khan]

On Wed, Aug 19, 2015 at 12:12 AM, Eli Zaretskii <eliz@gnu.org> wrote:
>> I used to use Windows, including the Windows console, as my primary
>> environment. It displays most of the European part of Unicode
>> allright, once you configure it to use a TrueType or OpenType font.
>
> Europe is but a small part of the world, and the fonts available for
> that on the Windows console are ugly and hard to read.

The Windows console has a two-stage mechanism for configuring the font.

First, the registry specifies a subset of fonts which can be used in
the console. Initially this subset only contains Lucida Console or
Courier New. As far as I can tell, Far East localizations add a
CJK-enabled font. Googling for “add windows console font” will give
you the exact steps.

After that, chosen fonts appear in the console properties dialog.

I used Andale Mono and subsequently Liberation Mono with great success.

>> Some applications (notably, ports of Unix utilities) have problems
>> displaying Unicode on the Windows console. That is a bug in those
>> applications.
>
> LOL.  I have yet to see a native Windows port of a significant Unix
> utility that doesn't have that "bug".

Ubiquity of a bug does not imply that it can or should go unfixed.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Richard Stallman wrote:

> The issue at hand is doc strings in the source code, not format strings.

True.  Sorry about the confusion.

> It's a bad practice to copy text from a help buffer into
> a doc string in the source code

That's a bit like saying it's bad practice to copy from a text file into a Lisp 
string.  Although such copying can lead to problems because arbitrary text can 
be misinterpreted in a Lisp string, as a practical matter it's often quite 
convenient to copy snippets of text files into Lisp strings or vice versa, and 
it often works well, either because users know the text doesn't have string 
escapes or delimiters, or because they repair any glitches that may arise.

And it's not just *Help* buffers.  I often cut and paste from documentation into 
code.  The Emacs info files have used curved quotes for some time, and it's 
helpful to have these work in doc strings too.  As a simple example, the Elisp 
manual says this:

  -- Function: eql value1 value2
      This function acts like ‘eq’ except when both arguments are
      numbers.  It compares numbers by type and numeric value,...

If I were to write a similar function I might copy some of this into the new 
function's doc string and then adapt it, like this:

   (defun eqfuzz (value1 value2)
     "Compare two values but use fuzzy comparison on numbers.
   This function acts like ‘eq’ except when both arguments are numbers....

This sort of editing is natural and convenient, and there's no compelling reason 
to prohibit it.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Stephen J. Turnbull]

Eli Zaretskii writes:

 > And once again, when all is said and done, and the [Windows-]
 > ported code handles Unicode characters correctly when writing to the
 > console, you are back at the console font problem I mentioned above:
 > anything outside the European locales is downright impossible, and
 > inside Europe you have only the ugly Lucida Console font.

Not to prejudge the overall issue, as there are many users of free
environments opposed to the curvely quotes, but seriously: you
complain that you use non-free software and its vendor doesn't support
a proposed Emacs feature?[1]  Surely this should not be a consideration
in Emacs development -- if Emacs features can be supported on Windows
or Mac, fine, do it, but Windows or Mac lack of support for an Emacs
feature is no reason to avoid installing that feature.

OTOH Alan's issue about the small glyph repertoire of the Linux
console is a valid concern, although the availability of capable
alternative consoles weakens that particular point to some degree.


Footnotes: 
[1]  And to add insult to injury, a feature that was in part bought,
paid for, and advocated by that vendor?

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Dmitry Gutov wrote:
 > So yes, I think it's valuable.

OK, thanks for reviewing; I installed the %q patch, and followed up with 
simplifications that it allows at the C level, mostly by using %qs in formats 
instead of uLSQM and uRSQM.  This makes the C code easier to read and so is a 
clear win.  Doing something similar in Elisp code makes the code harder to read, 
though, so I held off on that.

> two format sequences, one for opening quote, and one for closing. That
> would be more cumbersome, though.

Yes, it would be more cumbersome.  Instead of the current:

   (message "Press ‘?’ or ‘h’ for help, ‘q’ to quit")

we would have something like:

   (message "Press %<?%> or %<h%> for help, %<q%> to quit")

which is harder to read and is more error-prone.  This less-readable approach is 
used in GCC's source code, which must port to old-fashioned C++ compilers that 
lack support for multibyte characters in strings.  It is not needed for Emacs 
Lisp, and we shouldn't insist on it there.

>> Also, it wouldn't suffice for code like this:
>  >
>>      (insert (symbol-name type)
>>              (format " is a type (of kind ‘"))
>>      (help-insert-xref-button (symbol-name metatype)
>>                               'cl-help-type metatype)
>>      (insert (format "’)"))
>
> But this is not about diagnostic messages anymore, right? At the moment, IIRC,
> these situations are handled by substitute-command-keys (and there's no need to
> have curly quotes in the strings here).

At the moment these situations are handled by ‘format’.  Simply changing the 
code to use ‘substitute-command-keys’ and ASCII characters would not work in 
cases like the above, because (insert (substitute-command-keys "')")) would 
insert an apostrophe regardless of user text-quoting preference.  Of course 
there are workarounds but the workarounds are clumsy.  (Also, the 
ASCII-only-source approach would typically be a tad slower, which is annoying. :-)

> text -> (substitute-command-keys) -> text with "escaped" text prop
>       -> (translate-quotes) -> text with non-escaped straight quotes replaces

As mentioned above it's not obvious how to get that to work; even if we did so 
it'd be more complicated and error-prone, and a bit slower.  Hardly seems worth it.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Stephen J. Turnbull]

Óscar Fuentes writes:

 > Dmitry suggests this, and his comment about modern markup languages
 > restricting themselves to ASCII is something to think about.

Not really.  No chicken developed from that egg because there was no
chicken to lay the egg in the first place.  By and large programmers'
environments are deficient in respect of input methods, especially in
the U.S., and until a few years ago solid multilingual Unicode
environments weren't really available (and still aren't on Windows, if
I understand Eli's descriptions correctly).  So programmers (who
design markup languages) restrict themselves to ASCII-based markup.
It's only become reasonable to think about going beyond ASCII in the
last 5 years or so (if you want to maintain fairly general appeal).
And there's the counterexample of Xe[La]TeX, which in fact developed
for Mac, the most complete Unicode implementation available at the
time -- a single anecdote, but very suggestive IMHO.

Emacs is the perfect environment to experiment with *discoverable*
*multilingual* input methods.  AFAIK, they don't exist yet,
*anywhere*.  Apple is going backwards, even.  Microsoft doesn't have
them, either.  The proprietary technology is quite good -- within the
context of monolingual environments (which is where the money is, even
in Europe the number of companies where individuals need multilingual
environments is limited).  But they require effort for neophytes to
learn, and are less than useful for "inputting 'exotic' characters.
As far as I can tell, there's nothing better out there for free
software, either -- we're now on our fourth or fifth generation of new
input management frameworks for GNOME and/or KDE, and *still* the most
frequent n00b question on the Tokyo Linux Users Group[sic] is "I just
upgraded MyDistro and now I can't input Japanese in WhateverOffice".

My Chinese students and Buddhist scholar friends all use Macs because
it's very easy to switch among input methods (Chinese, Japanese, and
Sanskrit are radically different -- it's sort of possible to share an
input method between Chinese and Japanese, but it's very painful).
But all of these methods are monolingual, and must be learned
separately (or "taught", as most "learn" the user's habits, changing
priorities in the dictionaries and storing common sequences of words
for "predictive translation").

Emacs at least has Quail, giving language flexibility as good or
better than Apple, although the input methods themselves are static,
so aren't as user-friendly as the proprietary ones that "learn" the
users' habits.  And (one small step for Emacs, one giant step for
mankind) Quail methods are self-documenting (although again
discoverability needs to be improved for the purpose of "typing
'exotic' characters").

 > I admit that I'm intrigued by your plan about how this change will
 > initiate an evolution on Emacs input system that will make easier to
 > type exotic characters (defining "exotic" by "something that it is
 > infrequent in your daily usage.")

By giving people an itch they want to scratch.  Most people will just
cut'n'paste or add ad hoc keybindings for the characters they need.
Some people will do more, and sooner or later one of them will come up
with a much better way to do input methods.  It's not obvious to me
what that will be, and it's probably useless to ask Paul what it will
be too.

David K pointed out that there are some useful ideas in x-symbol.
That might be one place to look.

Also, besides input methods, it will likely lead to improvements in
other technologies such as searching (adding character classes of
"cognates" such as ` and ‘, for example -- this is useful for
repertoires like Japanese which has about a dozen variants on open
parenthesis more or less commonly used in text, as well as a pile of
numeral variants used for paragraph numbering, and the like).

Those opposed to the change will cry YAGNI, and that's true -- if you
live in an 8-bit world anyway, you just can't afford that kind of
redundancy.  But like it or not, the world is now mostly Unicode and
that will only increase.  Japanese is probably the most perverse
character set in existence, but I believe Chinese and Korean also have
similar issues of many classes of characters that have redundant
functionality, and it shows up in other places (eg, arrows and
emoticons).

 > Maybe describing the specific user-visible improvements that this
 > change will help to bring into reality would buy you more support.

The user-visible improvements have been described and are easily
visible to the eye desiring to see them.  Tastes just differ here; the
people who don't like the change see little to no improvement, and
IIUC Drew even considers it a clear step backward aesthetically.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Óscar Fuentes]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

[snip]

> Emacs is the perfect environment to experiment with *discoverable*
> *multilingual* input methods.

In my experiments with Unicode in source code, input was the least of
the concerns (as long as Emacs was used.) Display was the no-no part.
Not because Emacs bugs or limitations, by the way.

If the goal is to improve input for other scripts, I have no idea how
forcing curly quotes in Emacs' source code will help.

[snip]

>  > Maybe describing the specific user-visible improvements that this
>  > change will help to bring into reality would buy you more support.
>
> The user-visible improvements have been described and are easily
> visible to the eye desiring to see them.

Maybe I skipped the post that described the improvements, but what I've
seem so far is "because it is modern", "because beginners think our
current style is weird", "because other GNU projects use it", "because
this will help improve our Unicode support"... Am I missing something?

[snip]

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Óscar Fuentes]

Paul Eggert <eggert@cs.ucla.edu> writes:

> Alan Mackenzie wrote:
>
>> I think you know full well what I mean by "working character",
>
> No, actually, I don't.  From my point of view "working character" is a
> slippery notion that mutates when we try to pin it down.  Perhaps we
> should stick with "non-ASCII character"; that's clear.
>
>> Spanish keyboards DO have "Ó", for any sensible value of "have".
>
> Only if properly configured.  If you use these keyboards with the
> wrong settings, Compose ' O won't work, and there's no "Ó" key on the
> keyboard so you'll be stuck.  In these respects "Ó", "‘" and "’" are
> all in the same category on a Spanish keyboard.  And this is OK.

Paul, as an Spaniard with 30+ years of experience with computers, I can
assure you that accented letters is a non-issue since a *long* time ago.
I can't remember having issues with accents since I'm using PCs, that's
20+ years. The only way you can have problems with accented letters is
if you lie to the OS saying that your keyboard is not Spanish.

However, I'm lost if you ask me how to input "‘" and "’" (see, I
copy&pasted from your text). Probably I would use the Emacs Unicode
facilities looking for something that looks like those chars, and hope I
arrive at the actual chars and not at something with the same looks but
that's not quite the same.

[snip]

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Stephen J. Turnbull]

Óscar Fuentes writes:

 > Maybe I skipped the post that described the improvements, but what
 > I've seem so far is "because it is modern",

I think you misread.  The point is that modernizing projects are
already doing it and the sky didn't fall, not that calling a feature
"modern" makes it attractive.

 > "because beginners think our current style is weird",

Indeed, they do.  So do a lot of people who are way beyond beginners.

 > "because other GNU projects use it",

Indeed, they do.

 > "because this will help improve our Unicode support"...

I think it will, as I gather Paul does.

 > Am I missing something?

"It improves readability."  But that's a matter of taste.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Andreas Schwab]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> Óscar Fuentes writes:
>
>  > "because beginners think our current style is weird",
>
> Indeed, they do.  So do a lot of people who are way beyond beginners.

Should we switch to `...` then ?

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Nicolas Richard]

Paul Eggert <eggert@cs.ucla.edu> writes:
> Only if properly configured.  If you use these keyboards with the
> wrong settings, Compose ' O won't work, and there's no "Ó" key on the
> keyboard so you'll be stuck.  In these respects "Ó", "‘" and "’" are
> all in the same category on a Spanish keyboard.  And this is OK.

I don't know about Spanish keyboards.

On a french/belgian keyboard, however, some accented letters have their
own key (é è à ù), and for the rest of them we have dead keys. So for Ó,
I have <dead-acute> O.

The curly quotes OTOH are inconvenient to type for me. In emacs I now
know I can use "C-x 8 [" but that's 6 keys to press (numbers are shifted keys
unless I move to the numpad, and opening square bracket is obtained by
pressing AltGr and another key). I don't know yet how to type it outside
emacs. M-a M-k. I just found out how to type them outside emacs:
<compose> < ' is ‘ and <compose > ' is ’.

-- 
Nico

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Dmitry Gutov]

On 08/19/2015 09:28 AM, Paul Eggert wrote:

> OK, thanks for reviewing; I installed the %q patch, and followed up with
> simplifications that it allows at the C level, mostly by using %qs in
> formats instead of uLSQM and uRSQM.  This makes the C code easier to
> read and so is a clear win.  Doing something similar in Elisp code makes
> the code harder to read, though, so I held off on that.

Please don't take it as a review: I only verified that it corresponded 
to my proposal. But if you're not going to use the proposal as stated, 
and are still keeping curlies in Elisp source code, then, from where I'm 
standing, the new format sequence is not exactly justified.

> we would have something like:
>
>    (message "Press %<?%> or %<h%> for help, %<q%> to quit")

That looks quite nice to me.

> which is harder to read and is more error-prone.

The %qs approach should be the least error-prone.

> At the moment these situations are handled by ‘format’.  Simply changing
> the code to use ‘substitute-command-keys’ and ASCII characters would not
> work in cases like the above, because (insert (substitute-command-keys
> "')")) would insert an apostrophe regardless of user text-quoting
> preference.

Why? IIRC, when discussing a Lisp solution, you voted in favor of simple 
translation logic, one that didn't even check for pairings. 
substitute-command-keys could do the same.

>> text -> (substitute-command-keys) -> text with "escaped" text prop
>>       -> (translate-quotes) -> text with non-escaped straight quotes
>> replaces
>
> As mentioned above it's not obvious how to get that to work; even if we
> did so it'd be more complicated and error-prone, and a bit slower.
> Hardly seems worth it.

As I've explained several times, we'll need this (or maybe some other, 
comparable) complexity to solve related issues.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Stephen J. Turnbull]

Andreas Schwab writes:

 > Should we switch to `...` then ?

Looks like a deprecated Python idiom to me, so I'd say no.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Eli Zaretskii]

> From: Yuri Khan <yuri.v.khan@gmail.com>
> Date: Wed, 19 Aug 2015 10:45:32 +0600
> Cc: Alan Mackenzie <acm@muc.de>, Paul Eggert <eggert@cs.ucla.edu>, Brief Busters <dgutov@yandex.ru>, 
> 	Emacs developers <emacs-devel@gnu.org>
> 
> >> I used to use Windows, including the Windows console, as my primary
> >> environment. It displays most of the European part of Unicode
> >> allright, once you configure it to use a TrueType or OpenType font.
> >
> > Europe is but a small part of the world, and the fonts available for
> > that on the Windows console are ugly and hard to read.
> 
> The Windows console has a two-stage mechanism for configuring the font.
> 
> First, the registry specifies a subset of fonts which can be used in
> the console. Initially this subset only contains Lucida Console or
> Courier New. As far as I can tell, Far East localizations add a
> CJK-enabled font.

This is year 2015.  We are way past "localization" phase of the 1990s,
when certain features existed only in certain locales.  Features that
require installation of extra language packs, like Far-Eastern
localizations, and aren't available otherwise, cannot be relied upon.
They don't exist for all practical purposes.  You cannot tell your
users to install those localizations, because most of them won't.  The
result is that displaying CJK text on a Windows console only works in
CJK locales, and similarly with other scripts outside of Europe.  That
flies in the face of any decent multilingual environment such as
Emacs.

We need these features working out of the box, on any end-user's
machine.  Until then, they don't exist, and therefore Emacs in its
console mode cannot provide a decent multilingual environment on
Windows.

> I used Andale Mono and subsequently Liberation Mono with great success.

Good for you.  But this doesn't solve the problem for others.  Very
few people will invest a significant amount of their time into
tinkering with their systems.  A solution that relies on that will not
fly.

> >> Some applications (notably, ports of Unix utilities) have problems
> >> displaying Unicode on the Windows console. That is a bug in those
> >> applications.
> >
> > LOL.  I have yet to see a native Windows port of a significant Unix
> > utility that doesn't have that "bug".
> 
> Ubiquity of a bug does not imply that it can or should go unfixed.

But it does say something about the scale of the problem.  It's not
like a "non-buggy" port exists somewhere and you can tell people to
install it instead of the buggy one they have.  Fixing this is hard,
because Windows doesn't yet fully support the UTF-8 codepage, so any
program that uses 'char *' for text strings needs radical changes to
fix this "bug".  It's a small wonder that there are almost no programs
that have this fixed.

Your misrepresentations of these problems makes a disservice to people
who read them.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Eli Zaretskii]

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Cc: Yuri Khan <yuri.v.khan@gmail.com>,
>     acm@muc.de,
>     eggert@cs.ucla.edu,
>     emacs-devel@gnu.org,
>     dgutov@yandex.ru
> Date: Wed, 19 Aug 2015 14:19:17 +0900
> 
> > And once again, when all is said and done, and the [Windows-]
>  > ported code handles Unicode characters correctly when writing to the
>  > console, you are back at the console font problem I mentioned above:
>  > anything outside the European locales is downright impossible, and
>  > inside Europe you have only the ugly Lucida Console font.
> 
> Not to prejudge the overall issue, as there are many users of free
> environments opposed to the curvely quotes, but seriously: you
> complain that you use non-free software and its vendor doesn't support
> a proposed Emacs feature?

I didn't complain.  You are confusing me with someone else.  I have no
problem with these quotes, the way they are currently supported by
Emacs.  You won't find me arguing against them in this thread (or any
other thread).

What I did was describe the situation on Windows in this respect in
response to a specific post that IMO misrepresented it.  I believe
Emacs behavior when displaying curved quotes on the Windows console is
adequate, but that doesn't mean the problems we are facing there don't
exist, or can be dismissed or easily solved.  I reported one such
minor problem lately, and Paul fixed it in a way that I consider
satisfactory.

> Surely this should not be a consideration in Emacs development -- if
> Emacs features can be supported on Windows or Mac, fine, do it, but
> Windows or Mac lack of support for an Emacs feature is no reason to
> avoid installing that feature.

Red herring.  I never said nor did anything to the contrary.  I
actually wrote some (small part) of the code to support this feature.
You are confusing me with someone else.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Eli Zaretskii]

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Date: Wed, 19 Aug 2015 15:31:47 +0900
> Cc: emacs-devel@gnu.org
> 
> until a few years ago solid multilingual Unicode environments
> weren't really available (and still aren't on Windows, if I
> understand Eli's descriptions correctly)

Not quite.  Solid multilingual environments do exist on Windows, just
not for console (i.e. text-mode) programs.  GUI programs on MS-Windows
have any number of facilities for presenting fully functional
multilingual environments, and Emacs uses these facilities to a large
degree, both for keyboard input and for display.

> Emacs is the perfect environment to experiment with *discoverable*
> *multilingual* input methods.  AFAIK, they don't exist yet,
> *anywhere*.  Apple is going backwards, even.  Microsoft doesn't have
> them, either.

If you are implying that input methods don't exist on Windows, then
this is false: they do.

FWIW, I don't understand and don't share people's gripes about input
methods in Emacs.  I use them quite a lot, and actually prefer them to
whatever keyboards I have installed on my systems: the Emacs input
methods are easier to learn and modify, work the same on any system,
and there's an easy way of showing which key does what.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Marcin Borkowski]

On 2015-08-19, at 12:10, Nicolas Richard <youngfrog@members.fsf.org> wrote:

> emacs. M-a M-k. I just found out how to type them outside emacs:

For the record: instead of M-a M-k, I'd use C-x <backspace>.

Best,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Andreas Schwab]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> Andreas Schwab writes:
>
>  > Should we switch to `...` then ?
>
> Looks like a deprecated Python idiom to me, so I'd say no.

Everyone is using it, so it must be cool.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Stephen J. Turnbull]

Eli Zaretskii writes:

 > If you are implying that input methods don't exist on Windows, then
 > this is false: they do.

No, I'm saying that the people around me who have to do both Japanese
and Chinese input prefer Mac, largely because it's easier to configure
Mac with multiple input methods and switch among them when doing email
and the like.

That may just be that Macs are generally easier to configure without
paying a consulting firm $500 for the Power User course, of course.  I
don't have personal experience.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Stephen J. Turnbull]

Eli Zaretskii writes:

 > > Not to prejudge the overall issue, as there are many users of free
 > > environments opposed to the curvely quotes, but seriously: you
 > > complain that you use non-free software and its vendor doesn't support
 > > a proposed Emacs feature?
 > 
 > I didn't complain.  You are confusing me with someone else.

I apologize for the confusion, and wonder why you didn't say the same
thing that I did to that someone else.<wink/>

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Stephen J. Turnbull]

Andreas Schwab writes:

 > Everyone is using it, so it must be cool.

To borrow a line from Eli, you are confusing me with someone else.  I
never argued that, and I don't recall anybody in favor of the change
arguing it either.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > For inserting proper punctuation marks automatically when writing prose, 
  > you should check out 
  > http://www.emacswiki.org/emacs/TypographicalPunctuationMarks

If this is a good feature, it should not be relegated to a wiki.  It
should be available in Emacs and documented in the Emacs Manual.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I admit that curly quotes are nicer, but that's easily achievable: make
  > them appear on *Help* and other buffers that shows docstrings.

Haven't we already done that?

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > I admit that curly quotes are nicer, but that's easily achievable: make
  > > them appear on*Help*  and other buffers that shows docstrings.

  > That was done weeks ago.  This thread is about something else: quotes in 
  > diagnostics.

Perhaps the thread was initially about quotes in diagnostics, but a
few days ago the issue of doc strings seemed to come up.  That is why
I started posting.

Did I misunderstand?
-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > Couldn't we easily have a mode for use in Text mode, that turns ' into
  > > ‘ or ’ according to the context?  (Maybe we already do.)

  > Yes we do, in Emacs master.  It's Electric Quote mode.

I had a vague memory we had that, but people were conversing as if
it didn't exist.

							    You can try it out by 
  > putting this into your ~/.emacs file:

Would someone like to ask the person who criticized us
to try that and see if he finds Emacs better now that it has that feature?

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Emacs is the perfect environment to experiment with *discoverable*
  > *multilingual* input methods.

Hear, hear!

Maybe we should bind the INSERT key to a new facility for easily
discoverable insertion of characters.  If they fit, it could also
provide ways to insert other things -- buffers, files, the kill ring
and registers, effectively becoming a prefix key for all kinds of
insertion.

This would make the command overwrite-mode harder to type.  I don't
know how many users would mind that; perhaps we should conduct a poll.
INSERT INSERT could run overwrite-mode.

What do people think?

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Richard Stallman wrote:
> Perhaps the thread was initially about quotes in diagnostics, but a
> few days ago the issue of doc strings seemed to come up.  That is why
> I started posting.
>
> Did I misunderstand?

I think so.  Your first message about doc strings, dated Monday:

http://lists.gnu.org/archive/html/emacs-devel/2015-08/msg00608.html

was in reply to a Sunday message about diagnostics:

http://lists.gnu.org/archive/html/emacs-devel/2015-08/msg00566.html

The docstring changes were mostly done weeks ago and haven't changed much 
recently.  The diagnostics changes are within the past week: they were initiated 
by Bug#21222 and they kicked off this thread.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Richard Stallman wrote:
>    > putting this into your ~/.emacs file:
>
> Would someone like to ask the person who criticized us
> to try that and see if he finds Emacs better now that it has that feature?

We can do that after Emacs 25 comes out.  That feature isn't in Emacs 24.5, and 
it may be a bit much to ask him to try out the bleeding-edge master version.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Nicolas Richard wrote:
> I just found out how to type them outside emacs:
> <compose> < ' is ‘ and <compose > ' is ’.

Yes, that's a common combination, standard in GTK+ and usable in both Emacs and 
other apps when running under Gnome.  Possibly the combination works for Óscar too.

Óscar Fuentes wrote:
> The only way you can have problems with accented letters is
> if you lie to the OS saying that your keyboard is not Spanish.

Yes, and I've lied to the OS about that sort of thing with localized keyboards; 
it happens.  We can't expect Emacs to work well in every such misconfiguration.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Dmitry Gutov wrote:
> if you're ... still
> keeping curlies in Elisp source code, then, from where I'm standing, the new
> format sequence is not exactly justified.

%q does help at the C level.  I suppose we could remove %q's documentation, and 
not export it to Lisp.  But perhaps it'd be better to leave things alone and see 
what people actually use.  Developers who prefer ASCII-only formats and who 
don't mind their code not working in Emacs 24.5-and-earlier can do this:

   (format "Environment variable name %qs contains %qs" var "=")

Developers who prefer more-intuitive formats, or who want their code to work 
reasonably well in Emacs 24.5-and-earlier, can do this:

   (format "Environment variable name ‘%s’ contains ‘=’" var)

Both approaches work on Emacs master now.  As the main objection to curved 
quotes appears to be the hassle of typing them in some environments, if a 
developer doesn't mind that hassle then it should be OK to use them.

>> we would have something like:
>>
>>    (message "Press %<?%> or %<h%> for help, %<q%> to quit")
>
> That looks quite nice to me.

Do you like trigraphs too?!?  (Sorry, couldn't resist.  :-)  Whatever niceness 
it has is clearly trumped by the niceness of using quotation marks to represent 
quotation marks, and by the better compatibility of quotation marks when running 
in older Emacs versions.

> when discussing a Lisp solution, you voted in favor of simple
> translation logic, one that didn't even check for pairings.
> substitute-command-keys could do the same.

I suppose it could, yes.  I'll look into that.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Dmitry Gutov]

On 08/20/2015 01:21 AM, Paul Eggert wrote:

> Do you like trigraphs too?!?  (Sorry, couldn't resist.  :-)  Whatever

:)

> niceness it has is clearly trumped by the niceness of using quotation
> marks to represent quotation marks, and by the better compatibility of
> quotation marks when running in older Emacs versions.

I can buy the "compatibility" argument. This approach is basically 
predicated on using electric-quotes-mode, however, and that brings in 
curly quotes in docstrings.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Andreas Schwab]

Paul Eggert <eggert@cs.ucla.edu> writes:

> Nicolas Richard wrote:
>> I just found out how to type them outside emacs:
>> <compose> < ' is ‘ and <compose > ' is ’.
>
> Yes, that's a common combination, standard in GTK+ and usable in both
> Emacs and other apps when running under Gnome.

This has nothig to do with gnome.  It's a standard X11 feature that
works everywhere.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Óscar Fuentes]

Paul Eggert <eggert@cs.ucla.edu> writes:

[snip]

> Óscar Fuentes wrote:
>> The only way you can have problems with accented letters is
>> if you lie to the OS saying that your keyboard is not Spanish.
>
> Yes, and I've lied to the OS about that sort of thing with localized
> keyboards; it happens.

Maybe that's because you are not a typical user. In my decades-long
experience supporting non-techy users, I barely recall a case involving
a misconfigured keyboard.

> We can't expect Emacs to work well in every such misconfiguration.

My point was about the difficulty of typing those curly quotes. That's
on a properly configured keyboard. BTW, the X <compose> method (which
doesn't work on my Kubuntu 15.04 + USA layout international variant, but
that's maybe a misconfiguration caused by my efforts of making it work
on a sensible way) is an aberration, IMHO. It *always* takes me more
time to figure out which key combination gives the char that I want than
searching the web and paste it. And when I use some <compose>
combination, and manage to remember it (difficult) then it is broken on
the next OS upgrade.

I'm sure that Emacs can do better than <compose>. I *think* that it
already does better. What I don't know is how forcing curly quotes on
the source code can help.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Wolfgang Jenkner]

On Tue, Aug 18 2015, Paul Eggert wrote:

> If it helps, I think
> diagnostics are the last major component of getting Emacs quoting to
> follow common modern practice (controlled by user preference of
> course).

I'd think that "common modern practice" for GNU projects is described in
(info "(gettext) po/LINGUAS").

IIUC, the description there recommends that the usual "ugly" quotes
should be used in source files and translated at run time according to
the locale.

So, provided things are not messed up by a third party, for the en
locale you get curly quotes only if you set a variant locale like
en@quot or en@boldquot (after making sure that your systems supports
those locales).  E.g., bash provides such message files.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Paul Eggert wrote:
>> when discussing a Lisp solution, you voted in favor of simple
>> translation logic, one that didn't even check for pairings.
>> substitute-command-keys could do the same.
>
> I suppose it could, yes.  I'll look into that.

After looking into it I'm not so sure it's a good idea.  Although it simplifies 
substitute-command-string, quite a few docstrings quote with apostrophes only 
when quoting non-symbols, e.g., for rfc2368-mailto-prequery-index:

   "Describes the portion of the url between 'mailto:' and '?'."

and it's not right to turn these into right single quotation marks:

   "Describes the portion of the url between ’mailto:’ and ’?’."

Although we could change all these docstrings to grave or curved or double 
quoting and that would make Emacs a bit more consistent, this is not clearly an 
improvement and there still would be a problem with users' own docstrings.

This sort of thing is why Electric Quote mode doesn't transform isolated 
apostrophes, and there is something to be said for keeping 
substitute-command-keys somewhat consistent with that mode.

So I'm inclined to leave this alone, and stick with the less-aggressive approach 
that substitute-command-keys is currently using.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Dmitry Gutov]

On 08/19/2015 09:14 PM, Richard Stallman wrote:

> If this is a good feature, it should not be relegated to a wiki.  It
> should be available in Emacs and documented in the Emacs Manual.

No argument from me here (but I'm not the target audience, so I don't 
really know how valuable it is, or whether than implementation works well).

typopunc.el is authored by Oliver Scholz who seems to have copyright 
assignment signed, but the wiki page contains a lot of code snippets 
seemingly required to get the most out of the package.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > it may be a bit much to ask him to try out the bleeding-edge master version.

Even if he decides not to do the work to try it, he might feel touched
that we are concerned about users like him.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Would someone like to summarize the various alternatives now
being considered for quotation in format strings?

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Richard Stallman wrote:
> Would someone like to summarize the various alternatives now
> being considered for quotation in format strings?

Perhaps best illustrated with examples.

Here's what's implemented in master now (either works):

   (format "‘add-to-list’ can't use var ‘%s’; use ‘push’ or ‘cl-pushnew’" sym)

   (format "%q can't use var %qs; use %qs or %qs" "add-to-list" sym "push" 
"cl-pushnew")

Digraphs were proposed, but not implemented:

   (format "%<add-to-list%> can't use var %<%s%>; use %<push%> or 
%<cl-pushnew%>" sym)

There are lots of other possibilities of course, but these have been the main 
ones considered so far.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Wolfgang Jenkner wrote:
> I'd think that "common modern practice" for GNU projects is described in
> (info "(gettext) po/LINGUAS").
>
> IIUC, the description there recommends that the usual "ugly" quotes
> should be used in source files and translated at run time according to
> the locale.

gettext caters to C and that part of its manual makes sense for languages like 
C, because portable C programs can't easily put non-ASCII characters into string 
literals and C programmers don't have a practical alternative to putting "ugly" 
quotes of some sort into their string literals.  This sort of uglification isn't 
necessary for Emacs Lisp, though.

If Emacs were to use gettext, presumably it would wrap every English-language 
diagnostic in a call to gettext, even diagnostics that don't involve quotes. 
For example:

   (format (gettext "Buffer %S has a running process; kill it? ")
           (buffer-name (current-buffer)))
   (format (gettext "‘add-to-list’ can't use var ‘%s’") sym)
   ...

We'd also need ask the Translation Project to maintain a set of translations for 
all the Emacs diagnostics.  This would work regardless of the quoting style used 
in the source code, and would be a good project to do, if someone wanted to do 
it.  It'd be a much bigger project than fixing quoting, though, as the set of 
changes involved to Emacs would be more complicated than the above simple 
examples suggest.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Dmitry Gutov]

On 08/19/2015 05:05 PM, Stephen J. Turnbull wrote:

>   > Should we switch to `...` then ?
>
> Looks like a deprecated Python idiom to me,

You might want to look into modern documentation markup languages, then.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Dmitry Gutov]

On 08/19/2015 09:52 PM, Paul Eggert wrote:

> We can do that after Emacs 25 comes out.  That feature isn't in Emacs
> 24.5, and it may be a bit much to ask him to try out the bleeding-edge
> master version.

Would it be too much trouble for you to provide a version of Emacs built 
from master to some of your students who expressed the confusion about 
the quoting method?

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Dmitry Gutov wrote:
> Would it be too much trouble for you to provide a version of Emacs built from
> master to some of your students who expressed the confusion about the quoting
> method?

I could try that with new students, yes.  (The old ones have mostly moved on....)

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Date: Wed, 19 Aug 2015 14:16:03 -0400
> Cc: ofv@wanadoo.es, emacs-devel@gnu.org
> 
> Maybe we should bind the INSERT key to a new facility for easily
> discoverable insertion of characters.  If they fit, it could also
> provide ways to insert other things -- buffers, files, the kill ring
> and registers, effectively becoming a prefix key for all kinds of
> insertion.
> 
> This would make the command overwrite-mode harder to type.  I don't
> know how many users would mind that; perhaps we should conduct a poll.
> INSERT INSERT could run overwrite-mode.

If we do that, there should be a prominent indication that the INSERT
"discovery" mode was activated.  I frequently press that key by
accident, when I actually want to press "Delete" or "Home" that are
nearby.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Eli Zaretskii]

> From: Wolfgang Jenkner <wjenkner@inode.at>
> Date: Thu, 20 Aug 2015 15:37:10 +0200
> Cc: Óscar Fuentes <ofv@wanadoo.es>, emacs-devel@gnu.org
> 
> On Tue, Aug 18 2015, Paul Eggert wrote:
> 
> > If it helps, I think
> > diagnostics are the last major component of getting Emacs quoting to
> > follow common modern practice (controlled by user preference of
> > course).
> 
> I'd think that "common modern practice" for GNU projects is described in
> (info "(gettext) po/LINGUAS").
> 
> IIUC, the description there recommends that the usual "ugly" quotes
> should be used in source files and translated at run time according to
> the locale.

That description is for messages in locale-specific languages.  If
someone wants to work on infrastructure for translating Emacs messages
to other languages, then they should include in that infrastructure
facilities for converting the quotes to locale-specific conventions
(or maybe rely on the translators to replace the quotes as part of the
translated messages, as gettext-based projects do).

But as long as Emacs speaks to the users only in US English, it is IMO
wrong to replace the quotes by locale-specific ones, exactly for the
same reason it is wrong to sort multi-lingual text in a buffer using
locale-specific sorting rules: those locale-specific rules were
invented for sorting the locale's language(s), not for sorting
multi-lingual text.  E.g., it would be very weird for me to see a
message quoting, say, Chinese text with German-style quotes just
because I happen to be in the de_DE locale!

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Eli Zaretskii]

> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Thu, 20 Aug 2015 16:56:56 +0300
> Cc: yandros@gmail.com, emacs-devel@gnu.org
> 
> typopunc.el is authored by Oliver Scholz who seems to have copyright 
> assignment signed, but the wiki page contains a lot of code snippets 
> seemingly required to get the most out of the package.

At least some parts of that should be simply added to iso-transl.el.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

If we can't use the implementation that's in the wiki, we should
reimplement this.

  > typopunc.el is authored by Oliver Scholz who seems to have copyright 
  > assignment signed, but the wiki page contains a lot of code snippets 
  > seemingly required to get the most out of the package.

Snippets might be ignorable for copyright purposes.
Where did these snippets come from?  How big is each one?

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[David Kastrup]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
> If we can't use the implementation that's in the wiki, we should
> reimplement this.
>
>   > typopunc.el is authored by Oliver Scholz who seems to have copyright 
>   > assignment signed,

I think that would cover contributions he makes to Emacs or possibly
other GNU projects for which he has an assignment on file (once the code
is (c) FSF, of course it can be moved to other FSF projects).  But I
don't think that grabbing stuff he contributed to a third-party Wiki and
incorporating it into Emacs would be covered by it.

Better to just ask.

>   > but the wiki page contains a lot of code snippets seemingly
>   > required to get the most out of the package.
>
> Snippets might be ignorable for copyright purposes.
> Where did these snippets come from?  How big is each one?

-- 
David Kastrup

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Ricardo Wurmus]

Dmitry Gutov <dgutov@yandex.ru> writes:

> typopunc.el is authored by Oliver Scholz who seems to have copyright 
> assignment signed, but the wiki page contains a lot of code snippets 
> seemingly required to get the most out of the package.

I’m using typopunct.el and all I needed was this:

  (require 'typopunct)
  (typopunct-change-language 'english t)

It’s taking care of placing the typographically correct quotes and
dashes—such as these—when I type ‘'’ or a series of dashes.

When I don’t want this to happen I first hit ‘C-q’ before typing ‘'’,
‘"’, or ‘-’.  Modifying these characters depending on context is all
typopunct does by default, and without any additional configuration.

~~ Ricardo

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Marcin Borkowski]

On 2015-08-21, at 15:58, Ricardo Wurmus <rekado@elephly.net> wrote:

> Dmitry Gutov <dgutov@yandex.ru> writes:
>
>> typopunc.el is authored by Oliver Scholz who seems to have copyright 
>> assignment signed, but the wiki page contains a lot of code snippets 
>> seemingly required to get the most out of the package.
>
> I’m using typopunct.el and all I needed was this:
>
>   (require 'typopunct)
>   (typopunct-change-language 'english t)
>
> It’s taking care of placing the typographically correct quotes and
> dashes—such as these—when I type ‘'’ or a series of dashes.
>
> When I don’t want this to happen I first hit ‘C-q’ before typing ‘'’,
> ‘"’, or ‘-’.  Modifying these characters depending on context is all
> typopunct does by default, and without any additional configuration.
>
> ~~ Ricardo

I'm also using typopunct, though I am not very happy with it.  The
advantage is that it is easy to add rules for another language, and to
switch languages on the fly:

--8<---------------cut here---------------start------------->8---
(require 'typopunct)
(add-to-list 'typopunct-language-alist
	     `(polish ,(decode-char 'ucs #x201E) ; opening-double-q (outer)
		      ,(decode-char 'ucs #x201D) ; closing-double-q (outer)
		      )) ; closing-single-q (inner)
(setq-default typopunct-buffer-language 'polish)

(defun typopunct-toggle-language ()
  "Turn typopunct-mode on and toggle typopunct language between
`polish' and `english'."
  (interactive)
  (typopunct-mode 1)
  (message "Typopunct is on.  Language changed to %s."
	   (if (string= typopunct-buffer-language "english")
	       (typopunct-change-language "polish")
	     (typopunct-change-language "english"))))

(defun typopunct-toggle-or-toggle-language (prefix)
  "Toggle typopunct on or off; with C-u, toggle the language."
  (interactive "P")
  (if prefix (typopunct-toggle-language)
    (typopunct-mode 'toggle)
    (message (format "Typopunct is %s.%s"
		     (if typopunct-mode "on" "off")
		     (if typopunct-mode
			 (concat "  Current language is " (symbol-name typopunct-buffer-language) ".")
		       "")))))

(global-set-key (kbd "C-z C-t") 'typopunct-toggle-or-toggle-language)
--8<---------------cut here---------------end--------------->8---

The disadvantage is that it probably should be a global minor mode.
This is especially important when editing the minibuffer.  While I'm in
a minibuffer, I can hit C-z C-t to switch it on, but this is cumbersome
(and doesn't work in Isearch).

Also, typopunct does its work with dashes based on the character before
point and not last-command or last-command-event or something like
that.  Not that it is a bad thing per se, but I'm not sure whether this
is the best solution.

So, typopunct as it is is definitely not sufficient; at least, it should
work in the minibuffer and with Isearch.

Best,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Stephen J. Turnbull]

Eli Zaretskii writes:

 > But as long as Emacs speaks to the users only in US English, it is IMO
 > wrong to replace the quotes by locale-specific ones, exactly for the
 > same reason it is wrong to sort multi-lingual text in a buffer using
 > locale-specific sorting rules: those locale-specific rules were
 > invented for sorting the locale's language(s), not for sorting
 > multi-lingual text.

+1

Definitely unset POSIX_ME_HARDER here.  Multilingual applications like
Emacs should treat locales as sources of data to be used appropriately
or on user request, not as standards to be taken seriously.

The "locale" should be inferred from the containing text (or metadata,
such as the gettext language) whereever possible.  Of course for
things like numerical data or dates a user might prefer to override
with their native locale.  But by default Emacs should not do that.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Dmitry Gutov]

On 08/20/2015 04:55 PM, Paul Eggert wrote:
> Although it
> simplifies substitute-command-string, quite a few docstrings quote with
> apostrophes only when quoting non-symbols, e.g., for
> rfc2368-mailto-prequery-index:
>
>    "Describes the portion of the url between 'mailto:' and '?'."
>
> and it's not right to turn these into right single quotation marks:
>
>    "Describes the portion of the url between ’mailto:’ and ’?’."

Isn't it?

Doesn't keeping the straight quotes in the output goes against your 
stated goal of improving typography in Emacs?

> This sort of thing is why Electric Quote mode doesn't transform isolated
> apostrophes, and there is something to be said for keeping
> substitute-command-keys somewhat consistent with that mode.

Whenever EQM doesn't translated isolated quotes, but the user needs 
them, they'll have to learn entering them the long way.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Dmitry Gutov wrote:
> On 08/20/2015 04:55 PM, Paul Eggert wrote:
>> e.g., for rfc2368-mailto-prequery-index:
>>
>>    "Describes the portion of the url between 'mailto:' and '?'."
>>
>> and it's not right to turn these into right single quotation marks:
>>
>>    "Describes the portion of the url between ’mailto:’ and ’?’."

> Doesn't keeping the straight quotes in the output goes against your stated goal
> of improving typography in Emacs?

Although it may not be ideal, neutral quotes are adequate typography here, and 
are certainly better than transforming all four quotes into right single 
quotation marks.  And although Emacs diagnostics use directed quotes to quote 
symbols and other code, the situation is not as clear for quoting other things 
so I have been conservative and have generally left these other quotes alone if 
they are neutral.  It'd be fine with me if you were bolder and changed them to 
be directed.

> Whenever EQM doesn't translated isolated quotes, but the user needs them,
> they'll have to learn entering them the long way.

Electric Quote mode would leave the apostrophes unmodified in the above example. 
  A user who really needs the string to contain four right single quotation 
marks (something that ordinarily would be inappropriate) will need to enter the 
characters manually, yes.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Dmitry Gutov]

On 08/22/2015 03:19 AM, Paul Eggert wrote:

> Although it may not be ideal, neutral quotes are adequate typography
> here,

Have you seen typographical guides allow using different kinds of single 
quotes in the same document? Using both kinds seems to violate 
typographical style, as well as GNU guidelines, mentioned recently, that 
mandate using appropriate quotes based on user's locale.

> and are certainly better than transforming all four quotes into
> right single quotation marks.

It's better than two right quotation marks, but worse than using one 
left and one right. So we don't need to optimize the workflow (rules, 
translation, electric modes, etc) for the given example.

> And although Emacs diagnostics use
> directed quotes to quote symbols and other code, the situation is not as
> clear for quoting other things so I have been conservative and have
> generally left these other quotes alone if they are neutral.  It'd be
> fine with me if you were bolder and changed them to be directed.

I'd rather not become complicit in this sprawling empire of quotes, yet.

Just pointing out another inconsistency. This time it's you 
contemplating using different quotes for quoting different things.

Whereas it all started with "let's have good typography". The mistake, 
as I see it, is trying to use "real" quotes for markup in the first place.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Dmitry Gutov]

On 08/20/2015 10:32 PM, Paul Eggert wrote:

> Here's what's implemented in master now (either works):
>
>    (format "‘add-to-list’ can't use var ‘%s’; use ‘push’ or
> ‘cl-pushnew’" sym)

Regarding automatic translation in `format': do you anticipate someone, 
somewhere, having a file with "‘add-to-list’" in its name, someone else 
writing a package which uses `format' for file name manipulation, and 
the first person trying it and getting an error on the file in question?

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > No, I'm saying that the people around me who have to do both Japanese
  > and Chinese input prefer Mac, largely because it's easier to configure
  > Mac with multiple input methods and switch among them when doing email
  > and the like.

Would you like to tell us what command interface SmackOS uses for
this?  Let's think constructively about making this part of Emacs
better.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > If we do that, there should be a prominent indication that the INSERT
  > "discovery" mode was activated.  I frequently press that key by
  > accident, when I actually want to press "Delete" or "Home" that are
  > nearby.

If it is an ordinary prefix key, you will get an echo after a second,

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > But as long as Emacs speaks to the users only in US English, it is IMO
  > wrong to replace the quotes by locale-specific ones,

I agree.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I think that would cover contributions he makes to Emacs or possibly
  > other GNU projects for which he has an assignment on file (once the code
  > is (c) FSF, of course it can be moved to other FSF projects).  But I
  > don't think that grabbing stuff he contributed to a third-party Wiki and
  > incorporating it into Emacs would be covered by it.

You are right.  We should ask him to say that he considers these
to be contributions to Emacs -- and then assign@gnu.org should save
a copy of his email.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Dmitry Gutov wrote:

> Regarding automatic translation in `format': do you anticipate someone,
> somewhere, having a file with "‘add-to-list’" in its name

Emacs diagnostics that use quotes have never worked well when they quote a file 
name or any other string that contains quotes or newlines.  This is true 
regardless of whether diagnostics quote 'like this', "like this", `like this', 
‘like this’, or “like this”, like this:, etc.  It would be helpful to devise and 
implement a syntax that would let Emacs quote arbitrary strings consistently and 
reliably, presumably by using backslash escapes for troublesome characters. 
Perhaps %q could do that, which would be an argument for preferring %q when 
quoting arbitrary strings in diagnostics.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Dmitry Gutov]

On 08/22/2015 09:48 PM, Paul Eggert wrote:
> Dmitry Gutov wrote:
>
>> Regarding automatic translation in `format': do you anticipate someone,
>> somewhere, having a file with "‘add-to-list’" in its name
>
> Emacs diagnostics that use quotes have never worked well when they quote
> a file name or any other string that contains quotes or newlines.

I'm not talking about diagnostics, just about the function `format', 
which is used in different contexts.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Dmitry Gutov wrote:
> On 08/22/2015 09:48 PM, Paul Eggert wrote:
>> Dmitry Gutov wrote:
>>
>>> Regarding automatic translation in `format': do you anticipate someone,
>>> somewhere, having a file with "‘add-to-list’" in its name
>>
>> Emacs diagnostics that use quotes have never worked well when they quote
>> a file name or any other string that contains quotes or newlines.
>
> I'm not talking about diagnostics, just about the function `format', which is
> used in different contexts.

Sorry, I'm not understanding the distinction.  The standard diagnostics 
functions ‘message’, ‘error’, etc. all call ‘format’.  Any of these functions 
can run into trouble when quoting a file name that can contain arbitrary characters.

For example:

   (format "Moving file ‘%s’ to ‘%s’" src dest)
   (message "Moving file ‘%s’ to ‘%s’" src dest)
   (error "Cannot move file ‘%s’ to ‘%s’" src dest)

All three uses generate ambiguous output if SRC and DEST can contain arbitrary 
characters, and this is true even if we fiddle with the format strings’ quoting 
styles.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Dmitry Gutov]

On 08/23/2015 01:43 AM, Paul Eggert wrote:

> For example:
>
>    (format "Moving file ‘%s’ to ‘%s’" src dest)
>    (message "Moving file ‘%s’ to ‘%s’" src dest)
>    (error "Cannot move file ‘%s’ to ‘%s’" src dest)
>
> All three uses generate ambiguous output if SRC and DEST can contain
> arbitrary characters, and this is true even if we fiddle with the format
> strings’ quoting styles.

The difference is that ambiguity in the latter two uses can't really 
cause a program to lose correctness (unless someone is catching and 
parsing the error's message). An ambiguity in `format' can cause an 
unexpected error due to a program, say, being unable to open a file.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Dmitry Gutov wrote:

> An ambiguity in `format' can cause an unexpected error due to a
> program, say, being unable to open a file.

Yes, although that's a longstanding problem: it's not something that the recent 
quoting-style changes have introduced, and it's not something that we really 
need to address now.

That being said, we could attack the problem somewhat by having %qs not only 
quote its output, but also insert backslashes in front of each troublesome 
character (which should include the characters in the string "\n\\\"'‘’“”", 
say).  This would work only in output contexts where backslash escapes make 
sense, but that's reasonably common and anyway the output would be unambiguous, 
which is better than what we have now.

A possible downside of changing %q's behavior in this way is that it'll double 
backslashes in file names, and maybe MS-Windows users wouldn't like that.  For 
example, if NAME is the file name ‘C:\Documents and 
Settings\Eggert\Example.png’, then:

    (error "Cannot find image file %qs" name)

would output:

    Cannot find image file ‘C:\\Documents and Settings\\Eggert\\Example.png’

and the doubled backslashes could be a bit disconcerting.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Stephen J. Turnbull]

Richard Stallman writes:

 > Would you like to tell us what command interface SmackOS uses for
 > this?

Not after that bit of rudeness.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  >    (format "Moving file ‘%s’ to ‘%s’" src dest)
  >    (message "Moving file ‘%s’ to ‘%s’" src dest)
  >    (error "Cannot move file ‘%s’ to ‘%s’" src dest)

  > All three uses generate ambiguous output if SRC and DEST can
  > contain arbitrary characters, and this is true even if we fiddle
  > with the format strings’ quoting styles.

That is true -- but the problem case is unusual, and the harm done
when it happens is not a disaster.  Meanwhile, it is not clear how to
solve the problem 100%.

We know how to do quoting that is reliable.  print does that for strings.
But that looks bad in these error messages.  Do we really want to make them
ugly in .1% of the cases to avoid ambiguity in .001% of the cases,
which will confuse the user in only .00001% of the cases?

(Those figures are guesses but I expect they are in the ballpark.)

Dmitry Gutov <dgutov@yandex.ru> wrote:

  > An ambiguity in `format' can cause an 
  > unexpected error due to a program, say, being unable to open a file.

That would happen only if you're trying to parse the output and use
the result, and that seems like bad practice to me.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Dmitry Gutov]

On 08/23/2015 02:45 AM, Paul Eggert wrote:

> Yes, although that's a longstanding problem: it's not something that the
> recent quoting-style changes have introduced, and it's not something
> that we really need to address now.

Nothing longstanding there. Where before, (format "‘abc’") passed 
through the input unchanged, now it _might_ be changed, depending on 
some global variable. That can bite a programmer in the behind.

> That being said, we could attack the problem somewhat by having %qs not
> only quote its output, but also insert backslashes in front of each
> troublesome character (which should include the characters in the string
> "\n\\\"'‘’“”", say).  This would work only in output contexts where
> backslash escapes make sense, but that's reasonably common and anyway
> the output would be unambiguous, which is better than what we have now.

That entirely misses my point.

>     (error "Cannot find image file %qs" name)

Again, I'm not talking about error messages.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Dmitry Gutov]

On 08/23/2015 07:30 AM, Richard Stallman wrote:

> That is true -- but the problem case is unusual, and the harm done
> when it happens is not a disaster.  Meanwhile, it is not clear how to
> solve the problem 100%.

One of the ways to solve it is not to introduce it in the first place.

> We know how to do quoting that is reliable.  print does that for strings.
> But that looks bad in these error messages.  Do we really want to make them
> ugly in .1% of the cases to avoid ambiguity in .001% of the cases,
> which will confuse the user in only .00001% of the cases?

I don't consider using a format sequence to be ugly. And we're talking 
about an issue of correctness here.

> That would happen only if you're trying to parse the output and use
> the result, and that seems like bad practice to me.

`format' is routinely used for string manipulation.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Dmitry Gutov wrote:
> Nothing longstanding there. Where before, (format "‘abc’") passed through the
> input unchanged, now it _might_ be changed, depending on some global variable.
> That can bite a programmer in the behind.

Ah, it sounds like you're worried about misuses like (format filename), where 
FILENAME might contain arbitrary characters including curved quotes.  However, 
code like that has always been buggy, as FILENAME may include %% (or % followed 
by something else, of course) and the %-sequences would be misinterpreted too.

Bugs like that do exist on occasion, but the recent change to ‘format’ hasn't 
introduced significantly more opportunities for them.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Dmitry Gutov]

On 08/23/2015 12:48 PM, Paul Eggert wrote:

> Ah, it sounds like you're worried about misuses like (format filename),
> where FILENAME might contain arbitrary characters including curved
> quotes.  However, code like that has always been buggy, as FILENAME may
> include %% (or % followed by something else, of course) and the
> %-sequences would be misinterpreted too.

They can, but they usually don't. Or the caller could be wise and escape 
'%'.

On the other hand, I thought we wanted to encourage more usage of 
unicode everywhere, probably even including filenames.

I can imagine a filename with ‘’ in it without trying too hard: think of 
an MP3 file the name of which reflects the song's title. And take a 
classical piece, with a complex title. They have quotes on occasion.

It's not limited to filenames, those are just the first thing that came 
to mind.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Andreas Schwab]

Paul Eggert <eggert@cs.ucla.edu> writes:

> Bugs like that do exist on occasion, but the recent change to ‘format’
> hasn't introduced significantly more opportunities for them.

It's not a bug to double all % and expect format to do TRT.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  >  > Would you like to tell us what command interface SmackOS uses for
  >  > this?

  > Not after that bit of rudeness.

Apple deserves it (and much worse), since it tyrannizes computer
users every day.

I can't make you help with this.  If you don't want to, I hope someone
else will.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Here's what's implemented in master now (either works):

  >    (format "‘add-to-list’ can't use var ‘%s’; use ‘push’ or ‘cl-pushnew’" sym)

That's not a feature -- it is just a way you can use the old 'format'
function.

  >    (format "%q can't use var %qs; use %qs or %qs" "add-to-list" sym "push" 
  > "cl-pushnew")

I see no harm in that feature, but what is the difference between %q and %qs?

  > Digraphs were proposed, but not implemented:

  >    (format "%<add-to-list%> can't use var %<%s%>; use %<push%> or 
  > %<cl-pushnew%>" sym)

That seems much cleaner, since it does not require moving fixed parts
of the message out of the format string.

Does anyone have an objection to this feature?

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Richard Stallman wrote:
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>    > Here's what's implemented in master now (either works):
>
>    >    (format "‘add-to-list’ can't use var ‘%s’; use ‘push’ or ‘cl-pushnew’" sym)
>
> That's not a feature -- it is just a way you can use the old 'format'
> function.

The feature has been removed in master due to compatibility concerns, so this 
part of the discussion is moot now.  However, the idea was that ‘format’ 
requoted as per user preference, so that the resulting string would look like this:

    "`add-to-list' can't use var `foo'; use `push' or `cl-pushnew'"

if SYM was "foo" and the user preferred the grave quoting style.

I'm working on a patch to do this sort of requoting in a separate new function, 
which should resolve the compatibility concerns.

>    >    (format "%q can't use var %qs; use %qs or %qs" "add-to-list" sym "push"
>    > "cl-pushnew")
>
> what is the difference between %q and %qs?

Sorry, the first %q was a typo; it should have been %qs.

>    > Digraphs were proposed, but not implemented:
>
>    >    (format "%<add-to-list%> can't use var %<%s%>; use %<push%> or
>    > %<cl-pushnew%>" sym)
>
> That seems much cleaner, since it does not require moving fixed parts
> of the message out of the format string.
>
> Does anyone have an objection to this feature?

Yes, it's ugly and hard to read and maintain.  It's reminsicent of the trigraphs 
of ANSI C (and it's being introduced for the same reasons!) and in the long run 
it would suffer the same fate as trigraphs.

It's simpler when one can cut and paste text from text files into Lisp strings 
and vice versa, and have it just work.  Requiring escapes like %< and %> would 
work against this simplicity.  Obviously we can't always cut and paste arbitrary 
text, but we should be able to do so with single quotes without having to resort 
to a trigraph-like syntax.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Andreas Schwab wrote:
> It's not a bug to double all % and expect format to do TRT.

I see you reverted translation by ‘format’ of curved quotes in format strings. 
While this preserves compatibility better, it also causes some diagnostics to 
use curved quotes even if ‘text-quoting-style’ specifies an ASCII style, which 
likely would cause problems in traditional Emacs setups that prefer ASCII styles.

One possible fix is along the lines of one of Dmitry's earlier suggestions: add 
a function that translates a curved-style string to the style specified by 
‘text-quoting-style’, and have ‘message’, ‘error’, etc. invoke the new function 
on the format string before passing it to ‘format’.  This avoids affecting the 
behavior of ‘format’ itself and so should address the compatibility concerns 
that you and Dmitry noted.  I installed commit 
fbb5531fa11d13854b274d28ccd329c9b6652cfc in an attempt to do that.  Comments as 
usual are welcome.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Dmitry Gutov wrote:
> Just pointing out another inconsistency. This time it's you contemplating using
> different quotes for quoting different things.

On further thought you're right: we should make Emacs more consistent, i.e., 
prefer directed rather than neutral quotes in diagnostics and docstrings.  It's 
a bit more work now, but that's OK.  I'll look into doing that.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > I think you may have misunderstood what the controversy is about.
  > > It's not about adding or not adding support for curly quotes and curly
  > > ellipses in users' documents.  It's about whether or not to replace
  > > ` and ' in Emacs's source files' error messages and comments with
  > > curly quotes,

For overall ease of maintenance, I think we should discourage use of
curly quotes and other non-ASCII characters in comments within Emacs
sources and Emacs packages, because they are a _gratuitous_ impediment
to some developers' editing the sources.  That is, they cause trouble
for some, while providing no _practical_ benefit to anyone.

However, I see two exceptions where non-ASCII characters ought to be used.

* When giving someone's name, that is normally written with accented letters,
  it is reasonable to write the person's name correctly.

  If your terminal can't display those characters, you'll still see
  the name more or less, and it won't interfere with your
  understanding of the code (since the name doesn't explain the code).

* When talking about the use of specific non-ASCII characters, it is reasonable
  to write those characters literally.

  In this case, mentioning those characters is not gratuitous.

Does anyone suggest other exceptions?

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > helpful to have these work in doc strings too.  As a simple example, the Elisp 
  > manual says this:

  >   -- Function: eql value1 value2
  >       This function acts like ‘eq’ except when both arguments are
  >       numbers.  It compares numbers by type and numeric value,...

  > If I were to write a similar function I might copy some of this into the new 
  > function's doc string and then adapt it, like this:

  >    (defun eqfuzz (value1 value2)
  >      "Compare two values but use fuzzy comparison on numbers.
  >    This function acts like ‘eq’ except when both arguments are numbers....

Doc strings and Lisp manual entries are written in a very different
style; to start with a Lisp manual entry and edit it into a doc string,
you will have to do a lot of rewriting.

If you must also change the quotes, that's a tiny bit of extra work.

What you should really do is go to the source of the original function
and copy its doc string.  That's already in the style you want, so you
won't have to make changes (other than for your change in the
functionality), and as a small bonus you also won't have to change the
way quoting is written.

For doc strings in the source, we should stick to ASCII quotes.


-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

We could let the first argument to 'format'
be a symbol that specifies how to transform the format string.
Then the second argument would be the format string.

This feature would naturally propagate to 'message' and 'error'.

So you would write

  (format 'ascii-quotes
          "`add-to-list' can't use var `%s'; use `push' or `cl-pushnew'" sym)

and it would convert the quotes according to the display.

Or

  (format 'single-quotes
          "'add-to-list' can't use var '%s'; use 'push' or 'cl-pushnew'" sym)

and it would convert the quotes according to the display,
treating odd-numbered singlequotes as open-quotes.

You can take your choice, because either one will DTRT on all displays.
The specified symbol's 'format-transform' property should be a
function that makes the change.  This way, many such transformations
can be implemented.  A list of transformation symbols would mean to
make several of them.

This avoids all the disadvantages you were talking about.


-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > >    >    (format "%<add-to-list%> can't use var %<%s%>; use %<push%> or
  > >    > %<cl-pushnew%>" sym)
  > >
  > > That seems much cleaner, since it does not require moving fixed parts
  > > of the message out of the format string.
  > >
  > > Does anyone have an objection to this feature?

  > Yes, it's ugly and hard to read and maintain.

I don't think it is ugly.

						   It's reminsicent of
  > the trigraphs of ANSI C (and it's being introduced for the same
  > reasons!)

Not at all the same.

Trigraphs substituted for specific characters in input.
These would generate varying characters in output.

However, my suggestion in the other message is more general
than all of these.


-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

On 08/24/2015 01:20 PM, Richard Stallman wrote:
> We could let the first argument to 'format'
> be a symbol that specifies how to transform the format string.

Something like that could work, yes.  For example, instead of the 
current master's:

   (format-message FORMAT ARGS)

we could have:

   (format 'message FORMAT ARGS)

where 'message means "Format appropriately for message functions" and 
translates curved quotes to the user's preferred style.  We can add 
support for other symbols or lists of symbols in place of 'message.

format-message does have an advantage, though: it simplifies 
backward-compatibility implementation.  For example, code intended to 
work reasonably well in older Emacs can do this:

   (unless (fboundp 'format-message) (defalias 'format-message 'format))

and doing this sort of thing would be somewhat trickier with (format 
'message ...).  So perhaps we should keep the (format-message ...) 
convenience function even if we also add (format 'message ...).

> curly quotes and other non-ASCII characters in comments within Emacs ... [provide] no_practical_  benefit to anyone.

This doesn't match my experience.  In the past few weeks I've edited 
hundreds of diagnostics and docstrings in Elisp files, and in my 
environment curved quotes are significantly more practical than %< and 
%> would be.  With %< and %>, I'd have to hit three or four keys to type 
a quote, and the digraphs would make it harder to quickly judge string 
length and alignment.  In my environment curved quotes are also 
significantly better in strings than grave accent and apostrophe, as I 
can type curved quotes with single unshifted keystrokes, as compared to 
having to hit three keys to type either grave accent or apostrophe.  
Emacs should support environments that make curved quotes this easy; it 
shouldn't fight against them.

Perhaps for other users and other environments %< and %> are better, and 
so it'd be OK to add support for %< and %> in format strings, and I can 
volunteer to do that if you think it'd be helpful.  However, Emacs 
shouldn't *force* people to use %< and %> to get this functionality: 
there should be an easy way to quote intuitively, with quote characters 
standing for themselves in typical usage.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

I wrote:

> [Dmitry is] right: we should make Emacs more consistent, i.e.,
> prefer directed rather than neutral quotes in diagnostics and docstrings.  It's
> a bit more work now, but that's OK.  I'll look into doing that.

I gave it a shot in master commit 68280c5ee9b87d874ffa7c111b3cac7e634cee22. 
There are a couple of followup patches, so that docstrings and diagnostics 
prefer directed to neutral quotes when quoting code and data.  Comments welcome.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Dmitry Gutov]

On 08/24/2015 11:20 PM, Richard Stallman wrote:

> For overall ease of maintenance, I think we should discourage use of
> curly quotes and other non-ASCII characters in comments within Emacs
> sources and Emacs packages, because they are a _gratuitous_ impediment
> to some developers' editing the sources.  That is, they cause trouble
> for some, while providing no _practical_ benefit to anyone.

I would rephrase that instead to simply disallow non-ASCII characters in 
the *syntax* of Emacs comments (C and Lisp). If someone wants to write a 
comment in their native, non-English language, that is, of course, fine.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Dmitry Gutov]

On 08/25/2015 10:11 AM, Paul Eggert wrote:

> I gave it a shot in master commit
> 68280c5ee9b87d874ffa7c111b3cac7e634cee22. There are a couple of followup
> patches, so that docstrings and diagnostics prefer directed to neutral
> quotes when quoting code and data.  Comments welcome.

I'd like to point out that since `format' doesn't translate quotes 
anymore, the "\\`%s\\'" example doesn't apply: we're unlikely to 
encounter beginning-of-string and end-of-string anchors in the messages 
displayed to the user.

With that in mind, why can't format-message translate ` and ' to 
curlies? Then we won't need the latter in the source code.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Dmitry Gutov wrote:
> With that in mind, why can't format-message translate ` and ' to curlies? Then
> we won't need the latter in the source code.

Thanks, this sounds like a good suggestion.  We couldn't do this earlier, 
because changing ‘format’ in this way would have too many compatibility issues, 
but now that we have split ‘format-message’ out, the basic idea should work and 
should lessen the hassle of converting messages.  It also should mean that we 
don't need %< or %> or even %q.  I'll look into it.

Although I'm with Stefan about the yuck factor with ‘format-message’, this does 
seem to be the sort of thing that we could all sort of live with, i.e., one of 
the better of an unpleasant set of options....

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  >   With %< and %>, I'd have to hit three or four keys to type 
  > a quote,

%< is three keys: Push and hold Shift  5  ,  (release shift)

By contrast, open-curly-quote is C-x 8 [ which is four keys.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > For overall ease of maintenance, I think we should discourage use of
  > > curly quotes and other non-ASCII characters in comments within Emacs
  > > sources and Emacs packages, because they are a _gratuitous_ impediment
  > > to some developers' editing the sources.  That is, they cause trouble
  > > for some, while providing no _practical_ benefit to anyone.

  > I would rephrase that instead to simply disallow non-ASCII characters in 
  > the *syntax* of Emacs comments (C and Lisp).

That is not a rephrasing.  It's a totally different position.

I'm talking about the contents of comments, not their syntax.
The syntax of Lisp and C comments is well defined
and it does not involve non-ASCII characters.

I'm sayin comments shouldn't contain any non-ASCII characters except
under special circumstances.  I gave two examples of special
circumstances that would justify non-ASCII characters.


  > If someone wants to write a 
  > comment in their native, non-English language, that is, of course, fine.

Not in the Emacs sources.  Comments in GNU packages should always be
in English.  See the GNU Coding Standards, node Comments.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Dmitry Gutov]

On 08/26/2015 01:11 AM, Richard Stallman wrote:

> That is not a rephrasing.  It's a totally different position.
>
> I'm talking about the contents of comments, not their syntax.
> The syntax of Lisp and C comments is well defined
> and it does not involve non-ASCII characters.

I mean the syntax of the comments' contents. Which is what's being 
discussed here.

The way identifiers are quoted inside comments or docstrings is a part 
of that syntax.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Stefan Monnier]

> Thanks, this sounds like a good suggestion.  We couldn't do this earlier,
> because changing ‘format’ in this way would have too many compatibility
> issues, but now that we have split ‘format-message’ out, the basic idea
> should work and should lessen the hassle of converting messages.  It also
> should mean that we don't need %< or %> or even %q.  I'll look into it.

IOW, we don't need format-message at all.  Instead we can just have
a "prettify" function which can be used on the output of `format' (or on
any other string we like) and which uses some heuristic to turn `...'
into its ‘...’ equivalent.
I'm not sure I like the idea of a heuristic, but it does sound better
than an "almost like ‘format’ but different".


        Stefan

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Stefan Monnier wrote:

> IOW, we don't need format-message at all.  Instead we can just have
> a "prettify" function which can be used on the output of `format' (or on
> any other string we like) and which uses some heuristic to turn `...'
> into its ‘...’ equivalent.

I had already put something along those lines in master (the prettify function's 
name was ‘internal--text-restyle’), but despite hacking around with it for a 
while I didn’t find a good independent use for it.

Every use was of the form (format (prettify STRING) ARGS).  The prettify 
function wasn't useful as postpass to ‘format’ because that would have typically 
mishandled examples like (prettify (format "Invalid `%s'" xxx)) where XXX 
contains quotes.  Nor was it useful as a prepass or postpass to 
‘substitute-command-keys’, for similar reasons.

Plus, even with prettify split out, we need a format-message function anyway, if 
only as a convenience, because it's so often needed in examples like (apply 
#'format-message fmt args).  Since there was a demonstrated need for 
format-message but not for prettify (and before reading your abovequoted email) 
I redid the code to export the former and not the latter.  The new 
implementation avoids burdening the GC with two copies of the string, which is a 
minor performance win.

If there's a use case for the prettify function I'll be happy to resurrect it, 
of course.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Richard Stallman wrote:
>    >   With %< and %>, I'd have to hit three or four keys to type
>    > a quote,
>
> %< is three keys: Push and hold Shift  5  ,  (release shift)
>
> By contrast, open-curly-quote is C-x 8 [ which is four keys.

In another subthread Dmitry came up with a better suggestion which involves 
using plain ` and ' in format strings and Iinstalled a change to do that, so I 
think this point is moot now.  You can see his email at:

http://lists.gnu.org/archive/html/emacs-devel/2015-08/msg00893.html

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I mean the syntax of the comments' contents. Which is what's being 
  > discussed here.

  > The way identifiers are quoted inside comments or docstrings is a part 
  > of that syntax.

It is better not to use non-ASCII characters gratuitously.
Thus, better to write the quotes in comments with ASCII characters.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Jean-Christophe Helary]

> On Aug 27, 2015, at 03:51, Richard Stallman <rms@gnu.org> wrote:
> 
> It is better not to use non-ASCII characters gratuitously.
> Thus, better to write the quotes in comments with ASCII characters.

Only 2 natural languages on earth can use ASCII only. Emacs ought to adapt to that reality.

Jean-Christophe Helary

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[David Kastrup]

Jean-Christophe Helary <jean.christophe.helary@gmail.com> writes:

>> On Aug 27, 2015, at 03:51, Richard Stallman <rms@gnu.org> wrote:
>> 
>> It is better not to use non-ASCII characters gratuitously.
>> Thus, better to write the quotes in comments with ASCII characters.
>
> Only 2 natural languages on earth can use ASCII only. Emacs ought to
> adapt to that reality.

Huh?  Emacs has no problems working with non-ASCII characters.  But its
comment language is English and its "message catalog" is English,
period.  Changing the former does not make sense, and changing the
latter is not under discussion and would not change the untranslated
texts anyway.

-- 
David Kastrup

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Jean-Christophe Helary]

> On Aug 27, 2015, at 07:39, David Kastrup <dak@gnu.org> wrote:

>> Only 2 natural languages on earth can use ASCII only. Emacs ought to
>> adapt to that reality.
> 
> Huh?  Emacs has no problems working with non-ASCII characters.  But its
> comment language is English and its "message catalog" is English,
> period.  Changing the former does not make sense,

It does make plenty of sense.
You are confusing a natural language and a character set.
Comments are natural language. There are no technical reasons why they should be limited to such a limited character set.
Two Chinese developers who share code have no reason to write comments in English and limiting them to ascii to write Chinese is plain silly.

Jean-Christophe Helary 

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Eli Zaretskii]

> From: Jean-Christophe Helary <jean.christophe.helary@gmail.com>
> Date: Thu, 27 Aug 2015 07:53:15 +0900
> 
> 
> Two Chinese developers who share code have no reason to write comments in English and limiting them to ascii to write Chinese is plain silly.

We are not talking about code private to 2 Chinese developers.  We are
talking about the Emacs core code that is common to all Emacs
developers.  It is plain silly to suggest that the core code, read and
written by developers all over the world, could be in any language
except one that is common to all of them.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Alexis]

Jean-Christophe Helary <jean.christophe.helary@gmail.com> writes:

>> On Aug 27, 2015, at 03:51, Richard Stallman <rms@gnu.org> 
>> wrote:
>> 
>> It is better not to use non-ASCII characters gratuitously. 
>> Thus, better to write the quotes in comments with ASCII 
>> characters.
>
> Only 2 natural languages on earth can use ASCII only.

Out of interest, what's the natural language other than English 
that can use ASCII only?


Alexis.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Jean-Christophe Helary]

> On Aug 27, 2015, at 11:44, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Jean-Christophe Helary <jean.christophe.helary@gmail.com>
>> Date: Thu, 27 Aug 2015 07:53:15 +0900
>> 
>> 
>> Two Chinese developers who share code have no reason to write comments in English and limiting them to ascii to write Chinese is plain silly.
> 
> We are not talking about code private to 2 Chinese developers.  We are
> talking about the Emacs core code that is common to all Emacs
> developers.  It is plain silly to suggest that the core code, read and
> written by developers all over the world, could be in any language
> except one that is common to all of them.

Sorry, I got carried away by other comments. Obviously, since Emacs is GNU and GNU program comments have to be in English there is no point arguing otherwise.

Jean-Christophe Helary 

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Jean-Christophe Helary]

> On Aug 27, 2015, at 11:44, Eli Zaretskii <eliz@gnu.org> wrote:
> 
> We are not talking about code private to 2 Chinese developers.  We are
> talking about the Emacs core code that is common to all Emacs
> developers.  It is plain silly to suggest that the core code, read and
> written by developers all over the world, could be in any language
> except one that is common to all of them.

On a side note, there are quite a number of standard words in English that come from other languages and that require non ascii characters. So I guess if limiting comments to English is OK, enforcing a limit to (the printable characters subset of) ascii on comments could be short-sighted (even if realistically, there should be few comments including "résumé" or "élite" etc.)

Also, regarding limitations to ascii in comments for Lisp and C (as referred to earlier), I just checked the 2005 committee draft for ISO/IEC 9899 as well as ANSI X3.226-1994 and I could not find anything of the sort. I'm not used to read specs so I may have missed something though.

And to reply to Alexis, it is a language spoken by 4000 people on a small island East of New Guinea: Rokotas.

Jean-Christophe Helary

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Werner LEMBERG]

>> Only 2 natural languages on earth can use ASCII only.
> 
> Out of interest, what's the natural language other than English that
> can use ASCII only?

Actually, it's two more languages, AFAIK: Latin and Indonesian.  The
latter needs (similar to English) additional accents for words of
non-Indonesian origin only.

Hawaiian almost fits, too.  However, it needs a special comma for a
glottal stop, at least in print.


    Werner

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 615 bytes --]

Alexis wrote:
> Out of interest, what's the natural language other than English that can use
> ASCII only?

English isn’t one of the two ASCII-only languages.  It's naïve to think that 
even American English is ASCII-only.

I was once told that the other ASCII-only natural language is Hawaiʻian.  Pretty 
funny, huh?

Anyway, two reasonable answers to the original question would be roman Swahili 
and modern Latin.  Classical Latin needs some non-ASCII characters; see, for 
example:

https://commons.wikimedia.org/wiki/File:I_littera_in_manuscripto.jpg

which I'm attaching for convenience.

[-- Attachment #2: I_littera_in_manuscripto.jpg --]
[-- Type: image/jpeg, Size: 35119 bytes --]

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Werner LEMBERG]

> And to reply to Alexis, it is a language spoken by 4000 people on a
> small island East of New Guinea: Rokotas.

OK, three more languages :-)


    Werner

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Werner LEMBERG]

> Hawaiian almost fits, too.  However, it needs a special comma for a
> glottal stop, at least in print.

Oops, I'm wrong.  It needs macrons over vowels.


    Werner

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Werner LEMBERG]

> Classical Latin needs some non-ASCII characters; see, for example:
>
> https://commons.wikimedia.org/wiki/File:I_littera_in_manuscripto.jpg
>
> which I'm attaching for convenience.

Interesting.  However, this is not what you can see on the common
all-uppercase inscriptions (cf. Capitalis and Capitalis monumentalis).
So it seems that ASCII is *sufficient* to write correct Latin.


    Werner

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[David Kastrup]

Werner LEMBERG <wl@gnu.org> writes:

>> Classical Latin needs some non-ASCII characters; see, for example:
>>
>> https://commons.wikimedia.org/wiki/File:I_littera_in_manuscripto.jpg
>>
>> which I'm attaching for convenience.
>
> Interesting.  However, this is not what you can see on the common
> all-uppercase inscriptions (cf. Capitalis and Capitalis monumentalis).
> So it seems that ASCII is *sufficient* to write correct Latin.

Yes and no.  Antique Latin is written differently these days, just like
Antique Greek (which gradually adopted a three-accent system into
Alexandrian papyri in the 2nd century AD even though the system had
already been invented in the Hellenistic period in the 3rd century BC by
Aristophanes in Byzantium).

Writing correct Antique Greek _requires_ the use of the 3-accent system
even though nobody in Antique Greece ever used it.

-- 
David Kastrup

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 1421 bytes --]

Werner LEMBERG wrote:
> this is not what you can see on the common
> all-uppercase inscriptions (cf. Capitalis and Capitalis monumentalis).
> So it seems that ASCII is *sufficient*  to write correct Latin.

Yes and no.  Many people look at ancient uppercase Latin inscriptions and don’t 
notice the accents or the interpuncts or the distinction between LATIN CAPITAL 
LETTER I and LATIN EPIGRAPHIC LETTER I LONGA, because they’ve been programmed to 
think in the modern ASCII-only style for Latin.  Similarly, many of us spell the 
English word “naive” instead of “naïve” and reword other common English 
constructions that don't fit in ASCII, and in that sense ASCII is “sufficient” 
to write correct English.  Still, if your job had been to carve the inscription 
on the first-century shrine of the Augustales at Herculaneum, which reads:

   AVGVSTÓ·SACR·
   A·A·LV́CIꟾ·A·FꟾLIꟾ·MEN·
   PROCVLVS·ET·IV́LIÁNVS·
   P · S ·
   DÉDICÁTIÓNE·DECVRIÓNIBVS·ET·
   AVGVSTÁLIBVS·CÉNAM·DEDÉRVNT

and if you’d declined to carve it that way on the grounds that you wanted to 
stick to ASCII, you probably would have been fired.

My source for the inscription text:

https://en.wikipedia.org/wiki/Apex_%28diacritic%29#Details

A photo of the inscription:

https://commons.wikimedia.org/wiki/File:Herculaneum_Collegio_degli_Augustali_Iscrizione.jpg

[-- Attachment #2: 800px-Piranesi-TombaScipioBarbatus.jpg --]
[-- Type: image/jpeg, Size: 122043 bytes --]

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Werner LEMBERG]

>> this is not what you can see on the common all-uppercase
>> inscriptions (cf. Capitalis and Capitalis monumentalis).  So it
>> seems that ASCII is *sufficient* to write correct Latin.
> 
> Yes and no.  [...]

Thanks a lot.  I've just learned something new.


    Werner

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Alexis]

Paul Eggert <eggert@cs.ucla.edu> writes:

> English isn’t one of the two ASCII-only languages.  It's naïve 
> to think that even American English is ASCII-only.

Good point! My native language is Australian English, not American 
English, and i would consider it incorrect to write 'naive' 
instead of 'naïve'; to me, the former is simply a kludge imposed 
by the limitations of ASCII.

Thanks to everyone who's commented on my question - it's been very 
educational. :-)


Alexis.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

I suspect we may be talking about two different questions.  I am
talking about code that is part of Emacs.  (I said so explicitly in my
first message about this.)

  > Two Chinese developers who share code have no reason to write
  > comments in English and limiting them to ascii to write Chinese is
  > plain silly.

If they want their changes to be installed in Emacs, they must write
the comments in English.  We do not accept changes with comments in
Chinese.

That rule applies to all GNU packages.  See the GNU Coding Standards,
section Commenting Your Work.

You seem to be thinking about code that is not meant to be part of
Emacs -- but there is no need for a rule about that code.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > On a side note, there are quite a number of standard words in
  > English that come from other languages and that require non ascii
  > characters.

Mais non, we do not normally write accents on the words "resume" or
"elite" in English.  Nor "naive", which in English can be used
for either sex.

  > Also, regarding limitations to ascii in comments for Lisp and C
  > (as referred to earlier), I just checked the 2005 committee draft
  > for ISO/IEC 9899 as well as ANSI X3.226-1994 and I could not find
  > anything of the sort.

You can't expect to find the rules for GNU package sources in those
places.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > https://commons.wikimedia.org/wiki/File:I_littera_in_manuscripto.jpg

Where could I find an explanation of those symbols?
(I even can't see what they are, in that image.)

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: [TRUNCATED MESSAGE 2408 171455] Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > on the first-century shrine of the Augustales at Herculaneum, which reads:

What do those long vertical bars mean?

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Richard Stallman wrote:

>    > https://commons.wikimedia.org/wiki/File:I_littera_in_manuscripto.jpg
>
> Where could I find an explanation of those symbols?
> (I even can't see what they are, in that image.)

That fragment is old Roman cursive.  Most (but not all) those symbols are Latin 
letters expressible in ASCII.  Here's a text transliteration:

   uobis · ujdetur · p · c · decernám
   prólátis · rebus ijs · júdicibus · n
   imponátur quj · jntrá rerum
   jncoháta · judicia · non · per
   defuturas · ignoro · fraudes · m
   multas · aduersus · quas · exc

The "·" is an interpunct; it separates words, and some would write "·" rather 
than " · " when transcribing.  The accent over some vowels is an apex, and 
indicates that the vowel is long.  The transliteration uses Unicode acute 
accents for apices, which is standard practice nowadays, as the Unicode doesn't 
have apices and the acute accent is the modern descendant of the apex.

You can find a longer discussion of this topic here:

http://avitus.alcuinus.net/schola_latina/litterae_en.php

Re: [TRUNCATED MESSAGE 2408 171455] Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Richard Stallman wrote:
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>    > on the first-century shrine of the Augustales at Herculaneum, which reads:
>
> What do those long vertical bars mean?

The long vertical bars in words like "FꟾLIꟾ" are accented capital "I" letters, 
transcribed as U+A7FE LATIN EPIGRAPHIC LETTER I LONGA.  The accent indicates a 
long "I".  Common practice was to put an apex (the accent mark) over long 
vowels, e.g., "V́" was an accented "V" (the modern "U").  However, "I" was an 
exception: instead of putting an apex over an "I", people just wrote a taller "I".

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Should Emacs use punctuation syntax for "·"?
Are there other uses where it should be part of a word?

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Richard Stallman wrote:
> Should Emacs use punctuation syntax for "·"?

It probably should, if it isn’t already. In English it’s often used as a 
decimal-point character, and so should be in the same category as an ordinary 
period.

> Are there other uses where it should be part of a word?

Sure, in Catalan “cel·la” is a single word and the “·” should be part of the word.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Date: Thu, 27 Aug 2015 21:41:24 -0400
> Cc: emacs-devel@gnu.org
> 
> Should Emacs use punctuation syntax for "·"?

We already treat it as [:punct:], AFAIR.  But syntax is not the same
as the character properties, because syntax can depend on the major
mode and its needs.  For example, if a character can be part of a
word, it should have the word syntax.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Eli Zaretskii]

> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Thu, 27 Aug 2015 19:02:24 -0700
> Cc: emacs-devel@gnu.org
> 
> Richard Stallman wrote:
> > Should Emacs use punctuation syntax for "·"?
> 
> It probably should, if it isn’t already. In English it’s often used as a 
> decimal-point character, and so should be in the same category as an ordinary 
> period.
> 
> > Are there other uses where it should be part of a word?
> 
> Sure, in Catalan “cel·la” is a single word and the “·” should be part of the word.

Then at least in modes derived from Text mode, "·" cannot have the
punctuation syntax, I think.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Rasmus]

Paul Eggert <eggert@cs.ucla.edu> writes:

> Sure, in Catalan “cel·la” is a single word and the “·” should be part
> of the word.

Indeed it's quite frequently used in Catalan.  But I believe it's always
in the middle of a word (at least I haven't come across it elsewhere).
Can the syntax class be a function of surrounding characters (e.g. is
whitespace surrounding the characters)?

Rasmus

-- 
Hvor meget poesi tror De kommer ud af et glas isvand?

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Paul Eggert]

Rasmus wrote:
> Can the syntax class be a function of surrounding characters (e.g. is
> whitespace surrounding the characters)?

That wouldn't work for classical Latin, e.g., in "AVGVSTÓ·SACR" the "·" is not a 
letter even though it is surrounded by letters.

In general, whether a character is punctuation or a letter depends on the 
language.  "·" should be punctuation in English, arguably a space in classical 
Latin, and a letter in Catalan.  Is Emacs flexible enough to do that now?

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Rasmus]

Paul Eggert <eggert@cs.ucla.edu> writes:

> Rasmus wrote:
>> Can the syntax class be a function of surrounding characters (e.g. is
>> whitespace surrounding the characters)?
>
> That wouldn't work for classical Latin, e.g., in "AVGVSTÓ·SACR" the
> "·" is not a letter even though it is surrounded by letters.
>
> In general, whether a character is punctuation or a letter depends on
> the language.  "·" should be punctuation in English, arguably a space
> in classical Latin, and a letter in Catalan.  Is Emacs flexible enough
> to do that now?

But then you need to declare the language.  This is a buffer property.  In
some modes, such as org where you have a #+LANGUAGE cookie, or xml where
you have a meta tag it works, but in a plain old text buffer it's harder.
Maybe you could bind the variable, or it could depend on your current
ispell dictionary.  (In fact, I already need to explicitly add "['’]" to
CASECHARS in ispell-local-dictionary-alist to make hunspell accept words
like "it's").

Rasmus

-- 
C is for Cookie

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Marcin Borkowski]

On 2015-08-28, at 10:38, Rasmus <rasmus@gmx.us> wrote:

> Paul Eggert <eggert@cs.ucla.edu> writes:
>
>> Rasmus wrote:
>>> Can the syntax class be a function of surrounding characters (e.g. is
>>> whitespace surrounding the characters)?
>>
>> That wouldn't work for classical Latin, e.g., in "AVGVSTÓ·SACR" the
>> "·" is not a letter even though it is surrounded by letters.
>>
>> In general, whether a character is punctuation or a letter depends on
>> the language.  "·" should be punctuation in English, arguably a space
>> in classical Latin, and a letter in Catalan.  Is Emacs flexible enough
>> to do that now?
>
> But then you need to declare the language.  This is a buffer property.  In

Not really.  You seem to assume that no multilingual texts exist.  For
example, I work with texts containing parts in multiple languages
(usually, but not always, only two of them) on a daily basis.

OTOH, one might assume that they are properly marked up, e.g. in LaTeX.
(This is not always correct, but I might want to pay the price of
additional markup in such a case.)

> Rasmus

Best,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Eli Zaretskii]

> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Fri, 28 Aug 2015 01:32:40 -0700
> 
> In general, whether a character is punctuation or a letter depends on the 
> language.  "·" should be punctuation in English, arguably a space in classical 
> Latin, and a letter in Catalan.  Is Emacs flexible enough to do that now?

How do you define "the current language" in Emacs?  That's the main
problem here (as in any multi-lingual environment).

AFAIK, Emacs lacks infrastructure for providing language dependent
character properties, which differ for the same character.  However,
unless and until we come up with some useful definition of "the
current language", which itself might need some infrastructure, it
makes little sense to have language dependent properties.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Eli Zaretskii]

> From: Rasmus <rasmus@gmx.us>
> Date: Fri, 28 Aug 2015 10:38:36 +0200
> 
> > In general, whether a character is punctuation or a letter depends on
> > the language.  "·" should be punctuation in English, arguably a space
> > in classical Latin, and a letter in Catalan.  Is Emacs flexible enough
> > to do that now?
> 
> But then you need to declare the language.  This is a buffer
> property.

No, it isn't.  E.g., what is "the language" of the buffer you get by
typing "C-h H"?

> you have a meta tag it works, but in a plain old text buffer it's harder.
> Maybe you could bind the variable, or it could depend on your current
> ispell dictionary.

Spell-checker can get away with defining a superset of all the
languages it might be asked to spell-check.  But a feature Paul
mentions above cannot use that trick, for obvious reasons.

So the example of ispell is not useful here, IMO.

> (In fact, I already need to explicitly add "['’]" to CASECHARS in
> ispell-local-dictionary-alist to make hunspell accept words like
> "it's").

??? You mean like "it’", perhaps.  Because the apostrophe is already
in the dictionary's .aff file, and ispell.el gleans it from there.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Eli Zaretskii]

> From: Marcin Borkowski <mbork@mbork.pl>
> Date: Fri, 28 Aug 2015 10:47:39 +0200
> 
> OTOH, one might assume that [languages] are properly marked up,
> e.g. in LaTeX.

They aren't, not in Emacs, at least not in the basic Text mode.

If we want them marked with some property, we need to introduce such a
property and then add some code to do that automatically.  That code
is AFAICT not trivial at all, except in simple cases where the script
of the character unambiguously identifies the language (possible only
with scripts that support a single language).

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Marcin Borkowski]

On 2015-08-28, at 11:08, Eli Zaretskii <eliz@gnu.org> wrote:

>> From: Marcin Borkowski <mbork@mbork.pl>
>> Date: Fri, 28 Aug 2015 10:47:39 +0200
>> 
>> OTOH, one might assume that [languages] are properly marked up,
>> e.g. in LaTeX.
>
> They aren't, not in Emacs, at least not in the basic Text mode.

Of course.  OTOH, e.g. in LaTeX mode, we may have additional info, like
\usepackage[encoding]{inputenc}, which can be used by Emacs (AUCTeX, to
be more precise) to infer the encoding.  I imagine you could do
a similar thing with the language(s).  The problem of Text mode is
obviously still open.

> If we want them marked with some property, we need to introduce such a
> property and then add some code to do that automatically.  That code
> is AFAICT not trivial at all, except in simple cases where the script
> of the character unambiguously identifies the language (possible only
> with scripts that support a single language).

Yes, highly nontrivial.

Stanisław Barańczak, a Polish poet who died last year, amused himself by
many plays on words etc.  Among others, he wrote texts which were
_syntactically_ correct in two languages at the same time.  (AFAIR, he
did it for Polish/German and Polish/English combinations, so he got
bonus points for thus combining a Slavic language with a Germanic one.)

Best,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Rasmus]

Hi,

Eli Zaretskii <eliz@gnu.org> writes:

>> > In general, whether a character is punctuation or a letter depends on
>> > the language.  "·" should be punctuation in English, arguably a space
>> > in classical Latin, and a letter in Catalan.  Is Emacs flexible enough
>> > to do that now?
>> 
>> But then you need to declare the language.  This is a buffer
>> property.
>
> No, it isn't.  E.g., what is "the language" of the buffer you get by
> typing "C-h H"?

Also, as Marcin points out the language is really a "block" property, not
a buffer property.

>> (In fact, I already need to explicitly add "['’]" to CASECHARS in
>> ispell-local-dictionary-alist to make hunspell accept words like
>> "it's").
>
> ??? You mean like "it’", perhaps.  Because the apostrophe is already
> in the dictionary's .aff file, and ispell.el gleans it from there.

I have to add those characters to OTHERCHARS, sorry.  And I mean like
"it's" (keyboard quote).  I use Hunspell, maybe that's why?  If I don't do
this "it" and "'s" are seen as different words when I do ispell-word on
them.

Rasmus

-- 
⠠⠵

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Eli Zaretskii]

> From: Rasmus <rasmus@gmx.us>
> Cc: emacs-devel@gnu.org
> Date: Fri, 28 Aug 2015 11:53:13 +0200
> 
> >> (In fact, I already need to explicitly add "['’]" to CASECHARS in
> >> ispell-local-dictionary-alist to make hunspell accept words like
> >> "it's").
> >
> > ??? You mean like "it’", perhaps.  Because the apostrophe is already
> > in the dictionary's .aff file, and ispell.el gleans it from there.
> 
> I have to add those characters to OTHERCHARS, sorry.  And I mean like
> "it's" (keyboard quote).  I use Hunspell, maybe that's why?

I also use Hunspell.

> If I don't do this "it" and "'s" are seen as different words when I
> do ispell-word on them.

Look in the .aff file of your dictionary, there should be a WORDCHARS
entry there which includes the apostrophe.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Rasmus]

Eli Zaretskii <eliz@gnu.org> writes:

>> If I don't do this "it" and "'s" are seen as different words when I
>> do ispell-word on them.
>
> Look in the .aff file of your dictionary, there should be a WORDCHARS
> entry there which includes the apostrophe.

This is from my my en_US.aff:

     WORDCHARS 0123456789

And the package description of the package containing the file:

Nombre                  : hunspell-en
Versión                 : 2015.05.18-4
Descripción             : English hunspell dictionaries
Arquitectura            : any
Desarrollador principal : http://wordlist.aspell.net/dicts/

Rasmus

-- 
This message is brought to you by the department of redundant departments

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Eli Zaretskii]

> From: Rasmus <rasmus@gmx.us>
> Date: Fri, 28 Aug 2015 12:12:44 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> If I don't do this "it" and "'s" are seen as different words when I
> >> do ispell-word on them.
> >
> > Look in the .aff file of your dictionary, there should be a WORDCHARS
> > entry there which includes the apostrophe.
> 
> This is from my my en_US.aff:
> 
>      WORDCHARS 0123456789

Mine has this instead:

  WORDCHARS 0123456789'

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Rasmus]

Hi,

Eli Zaretskii <eliz@gnu.org> writes:

>> >> If I don't do this "it" and "'s" are seen as different words when I
>> >> do ispell-word on them.
>> >
>> > Look in the .aff file of your dictionary, there should be a WORDCHARS
>> > entry there which includes the apostrophe.
>> 
>> This is from my my en_US.aff:
>> 
>>      WORDCHARS 0123456789
>
> Mine has this instead:
>
>   WORDCHARS 0123456789'

When you download en_US from here it also does not have the apostrophe in WORDCHARS:

    http://wordlist.aspell.net/dicts/

It does have a line,

    ICONV ’ '

Rasmus

-- 
⠠⠵

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Eli Zaretskii]

> From: Rasmus <rasmus@gmx.us>
> Date: Fri, 28 Aug 2015 17:04:46 +0200
> 
> >> This is from my my en_US.aff:
> >> 
> >>      WORDCHARS 0123456789
> >
> > Mine has this instead:
> >
> >   WORDCHARS 0123456789'
> 
> When you download en_US from here it also does not have the apostrophe in WORDCHARS:
> 
>     http://wordlist.aspell.net/dicts/

Then it's broken, IMO.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Stefan Monnier]

> Indeed it's quite frequently used in Catalan.  But I believe it's always
> in the middle of a word (at least I haven't come across it elsewhere).
> Can the syntax class be a function of surrounding characters (e.g. is
> whitespace surrounding the characters)?

Sounds like the case of the ' character, then.
We handle this by giving it the syntax "w p" (in text-mode), so it's
considered as part of a word when it appears inside, but not elsewhere.

The distinction is most visible for `capitalize'.


        Stefan

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Fri, 28 Aug 2015 12:40:32 -0400
> Cc: emacs-devel@gnu.org
> 
> We handle this by giving it the syntax "w p" (in text-mode), so it's
> considered as part of a word when it appears inside, but not elsewhere.

Which AFAICS is not documented anywhere.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > Sure, in Catalan “cel·la” is a single word and the “·” should be part of the word.

  > Then at least in modes derived from Text mode, "·" cannot have the
  > punctuation syntax, I think.

Catalan takes priority over Latin, to be sure, if there is no good way
to handle both.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Sounds like the case of the ' character, then.
  > We handle this by giving it the syntax "w p" (in text-mode), so it's
  > considered as part of a word when it appears inside, but not elsewhere.

This won't directly work for middle-dot, when it is used in Latin with
no surrounding whitespace.

Perhaps the best thing is simply to write Latin using spaces
rather than middle-dots.  That is also correct.

But in general it would be good to label parts of text with a language
so that various features can automatically do the right thing: case
conversion, spell checking, and more.  Also character syntax
for middle-dot and other things.


-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Stefan Monnier]

>> We handle this by giving it the syntax "w p" (in text-mode), so it's
>> considered as part of a word when it appears inside, but not elsewhere.
> Which AFAICS is not documented anywhere.

No, indeed, it's a quick&ugly hack, which I'm not sure I want to
advertize too loudly.


        Stefan

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: rasmus@gmx.us,  emacs-devel@gnu.org
> Date: Sat, 29 Aug 2015 10:50:37 -0400
> 
> >> We handle this by giving it the syntax "w p" (in text-mode), so it's
> >> considered as part of a word when it appears inside, but not elsewhere.
> > Which AFAICS is not documented anywhere.
> 
> No, indeed, it's a quick&ugly hack, which I'm not sure I want to
> advertize too loudly.

Then how are we supposed to handle similar issues, if no one else
knows this, and never will?

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Stefan Monnier]

> Then how are we supposed to handle similar issues, if no one else
> knows this, and never will?

By designing a better solution, I guess.


        Stefan

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Sat, 29 Aug 2015 22:00:35 -0400
> Cc: rasmus@gmx.us, emacs-devel@gnu.org
> 
> > Then how are we supposed to handle similar issues, if no one else
> > knows this, and never will?
> 
> By designing a better solution, I guess.

I'm not sure I understand: are you saying that it is fundamentally
wrong or unclean to have a syntax category for word-constituent
characters that cannot appear at word beginning or end?  If so, please
explain why you think so, because this situation happens with many
characters in human languages, and is not really different from other
similar syntax categories.

If the idea is OK, and only its current implementation is not clean,
then I see no reason to refrain from documenting the Lisp-level
feature, because it will remain unchanged even when the implementation
will be cleaned up.

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[David Kastrup]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Stefan Monnier <monnier@iro.umontreal.ca>
>> Date: Sat, 29 Aug 2015 22:00:35 -0400
>> Cc: rasmus@gmx.us, emacs-devel@gnu.org
>> 
>> > Then how are we supposed to handle similar issues, if no one else
>> > knows this, and never will?
>> 
>> By designing a better solution, I guess.
>
> I'm not sure I understand: are you saying that it is fundamentally
> wrong or unclean to have a syntax category for word-constituent
> characters that cannot appear at word beginning or end?  If so, please
> explain why you think so, because this situation happens with many
> characters in human languages, and is not really different from other
> similar syntax categories.
>
> If the idea is OK, and only its current implementation is not clean,
> then I see no reason to refrain from documenting the Lisp-level
> feature, because it will remain unchanged even when the implementation
> will be cleaned up.

For the record: LilyPond's definition of the lexical category "word" is
any sequence of ASCII letters and arbitrary non-ASCII (multibyte)
characters interrupted by isolated hyphens and underlines.

c--d is a note c with a dash-separated accent - followed by a note d.
c-d is a word of its own.

There is a bit of history to this where LilyPond had too many different
definitions of "word" depending on its current lexical mode.

The respective definitions in the (Flex-defined) lexer are:

A               [a-zA-Z\200-\377]
WORD            {A}([-_]{A}|{A})*
COMMAND         \\{WORD}

The lexer is working on UTF-8 encoded bytes as input.  Whenever a
pattern accepts anything outside of the ASCII range, a checking routine
makes sure that only proper UTF-8 is passed on.

At any rate, it would be cool if words could be matched solely by syntax
table.  The "any non-ASCII character" bit might be impractical to
implement, but at least the word syntax inside of the ASCII range would
be nice.

-- 
David Kastrup

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Stefan Monnier]

> I'm not sure I understand: are you saying that it is fundamentally
> wrong or unclean to have a syntax category for word-constituent
> characters that cannot appear at word beginning or end?

No, I think the key question is rather whether it's OK for characters
like ' to have "w" as base syntax (which affects things like \<).

Maybe the "w p" syntax is actually a fine solution, but I'm not sure.
It was chosen as a quick&easy solution.


        Stefan

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: rasmus@gmx.us,  emacs-devel@gnu.org
> Date: Sun, 30 Aug 2015 21:33:29 -0400
> 
> > I'm not sure I understand: are you saying that it is fundamentally
> > wrong or unclean to have a syntax category for word-constituent
> > characters that cannot appear at word beginning or end?
> 
> No, I think the key question is rather whether it's OK for characters
> like ' to have "w" as base syntax (which affects things like \<).

So you are saying we should have a separate syntax category for such
characters?  If so, won't that have the downside that many regexps
that currently just use \w will need to use \w\|\X (where X is the new
category) instead?

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Stefan Monnier]

> So you are saying we should have a separate syntax category for such
> characters?

No.  I'm not suggesting any other solution.  I'm just explaining what
makes me slightly uneasy about the current solution.


        Stefan

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: rasmus@gmx.us,  emacs-devel@gnu.org
> Date: Mon, 31 Aug 2015 13:36:20 -0400
> 
> I'm not suggesting any other solution.  I'm just explaining what
> makes me slightly uneasy about the current solution.

And that thing that makes you uneasy, is it that 'w p' "affects things
like \<"?  That's about the only downside (or is it?) that you
mentioned, AFAICT.

IOW, what exactly is not okay about the current solution?

Re: [Emacs-diffs] master 9ce1d38: Use curved quotes in core elisp diagnostics

[Stefan Monnier]

> And that thing that makes you uneasy, is it that 'w p' "affects things
> like \<"?  That's about the only downside (or is it?) that you
> mentioned, AFAICT.

I'm pretty sure it affects various other parts of Emacs which just check
if the "base syntax" is "w".

Maybe it's an acceptable solution.  I just haven't spent the time to
convince myself either way.


        Stefan