Re: [Emacs-diffs] master 9d35bb8: Fix minor quoting problems in doc strings

Re: [Emacs-diffs] master 9d35bb8: Fix minor quoting problems in doc strings

[Dmitry Gutov]

On 05/20/2015 01:01 AM, Paul Eggert wrote:

> -with a space, for which the regexp is `\\` '.  See the source file for
> +with a space, for which the regexp is ‘\\` ’.  See the source file for
>
> -with a #, for which the regexp is `\\`#'.  See the source file for
> +with a #, for which the regexp is ‘\\`#’.  See the source file for
>
> -Match addresses of the style ``name%[stuff].'' when called with DELIM
> -of \"%\" and addresses of the style ``[stuff]name@[stuff]'' when
> +Match addresses of the style “name%[stuff].” when called with DELIM
> +of \"%\" and addresses of the style “[stuff]name@[stuff]” when
>
> -Used for file name completion.  Must not contain `'', `,' and `\"'
> +Used for file name completion.  Must not contain ‘'’, ‘,’ and ‘\"’

FWIW, these bits don't look quoting-method-agnostic to me.

> -  "Whether to show ``[N]'' for the Nth item up to 10.
> +  "Whether to show “[N]” for the Nth item up to 10.

The curly double quotes raise additional questions: would they be used 
everywhere instead of the escaped straight double-quotes (present in 
other places in your patch)? Do they have any special meaning, like the 
single quotes?

Re: [Emacs-diffs] master 9d35bb8: Fix minor quoting problems in doc strings

[Paul Eggert]

On 05/19/2015 03:30 PM, Dmitry Gutov wrote:
>
> FWIW, these bits don't look quoting-method-agnostic to me.
>

The problem they're trying to address is that one can't reasonably 
expect a regular expression to handle cases where grave accent or 
apostrophe is itself being quoted `like this'.  The simplest way out I 
could think of, for the few doc strings where this comes up, is to use a 
different quoting regime to quote the grave accent and/or apostrophe.  
(One can imagine more complicated schemes of course, but they seemed 
overkill.)  In this situation the patch preferred curved double quotes 
where the original used `` and '', and curved single quotes otherwise.

>> -  "Whether to show ``[N]'' for the Nth item up to 10.
>> +  "Whether to show “[N]” for the Nth item up to 10.
>
> The curly double quotes raise additional questions: would they be used 
> everywhere instead of the escaped straight double-quotes (present in 
> other places in your patch)?

I'm not proposing that, no.

> Do they have any special meaning, like the single quotes?

No, they're just replacements for `` and '', which also don't have any 
special meaning.

Another possibility, which would also be fine and would be simpler, 
would be to replace all curved double quotes in doc strings with 
straight double-quotes.  I could easily do that if you prefer; it's no 
big deal either way.

Re: [Emacs-diffs] master 9d35bb8: Fix minor quoting problems in doc strings

[Dmitry Gutov]

On 05/20/2015 02:18 AM, Paul Eggert wrote:

> The problem they're trying to address is that one can't reasonably
> expect a regular expression to handle cases where grave accent or
> apostrophe is itself being quoted `like this'.

Those two cases look simple (replace ` and the first matching ', the 
second ` would be skipped), but if the apostrophe were quoted, it'd have 
to be escaped: `\''.

And it's not very simple, but a regular expression can distinguish 
between an escaped and an unescaped character. Examples:

https://github.com/emacs-mirror/emacs/blob/9d35bb8d6518bb913ab08bace2af08963c003177/lisp/progmodes/js.el#L1698

https://github.com/emacs-mirror/emacs/blob/9d35bb8d6518bb913ab08bace2af08963c003177/lisp/progmodes/xref.el#L757

Though granted, that's a lot of backslashes.

> Another possibility, which would also be fine and would be simpler,
> would be to replace all curved double quotes in doc strings with
> straight double-quotes.  I could easily do that if you prefer; it's no
> big deal either way.

Not a big deal indeed, but consistency would be preferable IMHO.

Re: [Emacs-diffs] master 9d35bb8: Fix minor quoting problems in doc strings

[Paul Eggert]

Dmitry Gutov wrote:
> Those two cases look simple (replace ` and the first matching ', the second `
> would be skipped),

That would do the wrong thing in other places.  We have doc strings with stuff 
like this:

   This is like `(let ((v ,EXP)) ,EXPS) except that `v' is a new generated

which is why I'm leery about allowing ` within strings quoted `like this'.

> but if the apostrophe were quoted, it'd have to be escaped: `\''.

Which means the source would have to say "`\\''" and the backslash would have to 
be removed before showing the result to the user.

> And it's not very simple, but a regular expression can distinguish between an
> escaped and an unescaped character.

Ouch!  Let's keep things simple if we can.

> Not a big deal indeed, but consistency would be preferable IMHO.

OK, I installed commit f743819b57ef519109c1b9d520d358d19a197086 to do that.

Re: [Emacs-diffs] master 9d35bb8: Fix minor quoting problems in doc strings

[Dmitry Gutov]

On 05/20/2015 05:14 AM, Paul Eggert wrote:

> That would do the wrong thing in other places.  We have doc strings with
> stuff like this:
>
>    This is like `(let ((v ,EXP)) ,EXPS) except that `v' is a new generated
>
> which is why I'm leery about allowing ` within strings quoted `like this'.

If we're sure that quotes will be paired, a more advanced search 
function (or a longer regexp) could skip over the inner pairs as well.

However, note that the above example is simply missing a ' after the 
closing paren.

> Which means the source would have to say "`\\''" and the backslash would
> have to be removed before showing the result to the user.

Hmm, I guess so. It's not that terrible, though.

> Ouch!  Let's keep things simple if we can.

There's no hurry, but even if you used real curlies, where would be 
contexts when they could be nested as well, and the related code, 
ideally, would be able to handle it.

> OK, I installed commit f743819b57ef519109c1b9d520d358d19a197086 to do that.

Thank you.

Re: [Emacs-diffs] master 9d35bb8: Fix minor quoting problems in doc strings

[Dmitry Gutov]

On 05/20/2015 05:36 AM, Dmitry Gutov wrote:

> However, note that the above example is simply missing a ' after the
> closing paren.

Err no, sorry I'm being stupid. Apparently that initial backtick would 
have to be escaped.

Re: [Emacs-diffs] master 9d35bb8: Fix minor quoting problems in doc strings

[Paul Eggert]

>>    This is like `(let ((v ,EXP)) ,EXPS) except that `v' is a new generated

> If we're sure that quotes will be paired

Although the quotes that we want to curl are paired, there are other stray grave 
accents and apostrophes that we shouldn't curl.

> the above example is simply missing a ' after the closing paren.

Are you sure?  The first grave accent looks like a quasiquote, so there 
shouldn't be a closing apostrophe.  Those commas are clues that it's a quasiquote.

Here's my current draft regexp to catch the quotations we want to curl:

   `\\([^[:space:]'`‘’][^'`‘’]*\\)?'

A bit tricky, but could be worse.  Newlines are allowed, as some doc strings 
break these quotes across line boundaries (e.g., info node references).

Re: [Emacs-diffs] master 9d35bb8: Fix minor quoting problems in doc strings

[Yuri Khan]

On Wed, May 20, 2015 at 4:30 AM, Dmitry Gutov <dgutov@yandex.ru> wrote:

>> -Used for file name completion.  Must not contain `'', `,' and `\"'
>> +Used for file name completion.  Must not contain ‘'’, ‘,’ and ‘\"’
>
> FWIW, these bits don't look quoting-method-agnostic to me.

Why want quoting method agnosticism within text that is English and
known to be English in the foreseeable five or ten years? Curly quotes
are the semantically proper way of quoting things in English, and can
be easily degraded to the spacing grave accent and straight single
quote. Upgrading those to curly quotes, on the other hand, requires
solving a hard problem, and the solution likely involves backslashes.

Re: [Emacs-diffs] master 9d35bb8: Fix minor quoting problems in doc strings

[Dmitry Gutov]

On 05/20/2015 05:49 AM, Paul Eggert wrote:

> Although the quotes that we want to curl are paired, there are other
> stray grave accents and apostrophes that we shouldn't curl.

Probably. But if they're rare enough, having to escape won't be too 
annoying.

> Are you sure?  The first grave accent looks like a quasiquote, so there
> shouldn't be a closing apostrophe.  Those commas are clues that it's a
> quasiquote.

Indeed (I corrected myself later). If this kind of quasiquote is a 
popular kind of exception, we can handle it programmatically:

- If there's a opening paren after the backtick,
- And there's no apostrophe after the matching closing paren,

don't turn the backtick into the curly quote, and continue matching 
right after the closing paren.

If could be implemented in the form FACENAME in font-lock-keywords while 
keeping the matcher to be a regexp, but a custom matching function seems 
more appropriate, to catch any apostrophes (straight quotes) inside the 
form. See the patch.

> Here's my current draft regexp to catch the quotations we want to curl:
>
>    `\\([^[:space:]'`‘’][^'`‘’]*\\)?'
>
> A bit tricky, but could be worse.  Newlines are allowed, as some doc
> strings break these quotes across line boundaries (e.g., info node
> references).

Why not allow curly quotes inside? They don't introduce any syntax 
ambiguity.


diff --git a/lisp/help-mode.el b/lisp/help-mode.el
index f99e916..a0b7a6d 100644
--- a/lisp/help-mode.el
+++ b/lisp/help-mode.el
@@ -279,6 +279,19 @@ The format is (FUNCTION ARGS...).")
  \f
  (defvar bookmark-make-record-function)

+(defun help--match-quoted-code (limit)
+  (while (search-forward "`" limit t)
+    (let ((beg (point))
+          forward-sexp-function)
+      (when
+          (if (not (eq (char-after) ?\())
+              (re-search-forward "\\=\\([^[:space:]'`‘’][^'`‘’]*\\)?'" 
limit t)
+            (forward-sexp)
+            (eq (char-after) ?'))
+        (compose-region (1- beg) beg ?‘)
+        (compose-region (1- (point)) (point) ?’))))
+  nil)
+
  ;;;###autoload
  (define-derived-mode help-mode special-mode "Help"
    "Major mode for viewing help text and navigating references in it.
@@ -287,6 +300,7 @@ Commands:
  \\{help-mode-map}"
    (set (make-local-variable 'revert-buffer-function)
         'help-mode-revert-buffer)
+  (setq font-lock-defaults '(((help--match-quoted-code)) t))
    (set (make-local-variable 'bookmark-make-record-function)
         'help-bookmark-make-record))

Re: [Emacs-diffs] master 9d35bb8: Fix minor quoting problems in doc strings

[Stefan Monnier]

I've tried to stay away from this longish discussion because I have many
other things to do, but just to clarify my opinion overall:
I think that in the long term we should switch to using curly-quotes in
the actual source code.  How to get there is a good question, of course,
and I don't have a very good answer to it, but at least that means that
I think it's perfectly OK to start using curly-quotes in a few places,
and to start adding support for curly-quotes (e.g. to make it easier to
insert them, and to make *Help* buffer xrefs for them, to seamlessly
downgrade them to simple quotes on non-utf-8 terminals, ...), and
I think we should *also* add jit/font-lock support to display-upgrade
`...' into curly-quotes in those cases we deem "obvious enough".


        Stefan

Re: master 9d35bb8: Fix minor quoting problems in doc strings

[Glenn Morris]

Paul Eggert wrote:

> --- a/lisp/progmodes/f90.el
> +++ b/lisp/progmodes/f90.el
> @@ -1113,7 +1113,7 @@ For fixed format code, use `fortran-mode'.
>   indented line.
>  \\[f90-indent-subprogram] indents the current subprogram.
>  
> -Type `? or `\\[help-command] to display a list of built-in\
> +Type `?' or `\\[help-command]' to display a list of built-in\
[...]
> -  "Typing `\\[help-command] or `? lists all the F90 abbrevs.
> +  "Typing `\\[help-command]' or `?' lists all the F90 abbrevs.

Use of an unpaired ` was intentional, since that is what starts abbrevs
in F90 mode. `foo' isn't right here.

Re: master 9d35bb8: Fix minor quoting problems in doc strings

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 197 bytes --]

Glenn Morris wrote:
> Use of an unpaired ` was intentional, since that is what starts abbrevs
> in F90 mode. `foo' isn't right here.

Thanks for catching that.  I installed the attached to fix it.

[-- Attachment #2: 0001-Revert-doc-string-changes-to-f90.el.txt --]
[-- Type: text/plain, Size: 1347 bytes --]

From 0a3151c5ace3ae416ec7bef25b33753d5551a045 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Wed, 20 May 2015 22:16:53 -0700
Subject: [PATCH] Revert doc string changes to f90.el

Problem reported by Glenn Morris in:
http://lists.gnu.org/archive/html/emacs-devel/2015-05/msg00596.html
* lisp/progmodes/f90.el (f90-mode, f90-abbrev-start):
Revert recent changes to doc strings, as it's intended that they
use grave accent, not quote.
---
 lisp/progmodes/f90.el | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lisp/progmodes/f90.el b/lisp/progmodes/f90.el
index 74e3234..6264d3b 100644
--- a/lisp/progmodes/f90.el
+++ b/lisp/progmodes/f90.el
@@ -1113,7 +1113,7 @@ For fixed format code, use `fortran-mode'.
  indented line.
 \\[f90-indent-subprogram] indents the current subprogram.
 
-Type `?' or `\\[help-command]' to display a list of built-in\
+Type `? or `\\[help-command] to display a list of built-in\
  abbrevs for F90 keywords.
 
 Key definitions:
@@ -2267,7 +2267,7 @@ Leave point at the end of line."
 ;; Abbrevs and keywords.
 
 (defun f90-abbrev-start ()
-  "Typing `\\[help-command]' or `?' lists all the F90 abbrevs.
+  "Typing `\\[help-command] or `? lists all the F90 abbrevs.
 Any other key combination is executed normally."
   (interactive "*")
   (self-insert-command 1)
-- 
2.1.0

Re: [Emacs-diffs] master 9d35bb8: Fix minor quoting problems in doc strings

[Paul Eggert]

Stefan Monnier wrote:
> I think it's perfectly OK to start using curly-quotes in a few places,
> and to start adding support for curly-quotes (e.g. to make it easier to
> insert them, and to make*Help*  buffer xrefs for them, to seamlessly
> downgrade them to simple quotes on non-utf-8 terminals, ...), and
> I think we should*also*  add jit/font-lock support to display-upgrade
> `...' into curly-quotes in those cases we deem "obvious enough".

The master branch already supports downgrading to simple quotes.  Support for 
*Help* buffer xrefs is in the draft patches in <http://bugs.gnu.org/20385#295>, 
a patchset I just now published; rather than font-locking this patchset 
substitutes characters, an approach that worked better for me.

The main thing on my todo list is the first item on your list: making it easier 
to insert curved quotes.  This is Bug#20545, where the proposed patch 
<http://bugs.gnu.org/20545#5> needs a bit of hacking to get it to work well with 
electric-pair-mode.  I plan to look into that soon.  I'm thinking of renaming it 
to Electric Quote mode as "Electric Punct mode" may be a bit too ambitious.

Re: [Emacs-diffs] master 9d35bb8: Fix minor quoting problems in doc strings

[Dmitry Gutov]

On 05/21/2015 10:36 AM, Paul Eggert wrote:

> The main thing on my todo list is the first item on your list: making it
> easier to insert curved quotes.  This is Bug#20545, where the proposed
> patch <http://bugs.gnu.org/20545#5> needs a bit of hacking to get it to
> work well with electric-pair-mode.  I plan to look into that soon.  I'm
> thinking of renaming it to Electric Quote mode as "Electric Punct mode"
> may be a bit too ambitious.

Regarding this mode, how would one avoid the automatic replacement, in 
the cases when it's ambiguous, like the macroexp-let2 docstring? C-q?

Re: [Emacs-diffs] master 9d35bb8: Fix minor quoting problems in doc strings

[Paul Eggert]

Dmitry Gutov wrote:

> how would one avoid the automatic replacement, in the cases
> when it's ambiguous, like the macroexp-let2 docstring? C-q?

Yes, C-q ` will insert plain grave accent that is not turned into left single 
quotation mark.  Similarly, C-q ' will insert plain apostrophe even after 
unmatched left single quotation mark.  In practice it should be uncommon to type 
C-q, as almost all current uses of grave accent stand for open quote in doc strings.