Re: Escaping quotes in docstrings, Was: A simple solution to "Upcoming loss of usability ..."

[Paul Eggert <eggert@cs.ucla.edu>]

Dmitry Gutov wrote:
> On 06/28/2015 11:27 PM, Dmitry Gutov wrote:
>
>> Ok. But I'd rather push to the scratch/quote-escaping branch, if you
>> don't mind.
>
> Check it out.

I tried it out.  One thing I noticed right away is odd behavior that results 
from storing grave accent and apostrophe in the buffer and displaying them as 
curved quotes.  For example, run 'emacs -Q -nw', then read the documentation for 
the 'length' function and copy it to a new file /tmp/foo by typing "C-h f length 
RET C-x o C-x h M-w C-x 4 f /tmp/foo RET C-y C-s".  On the screen you'll see a 
buffer 'foo' containing curved quotes.  Now, revisit /tmp/foo by typing "C-x C-v 
RET".  The buffer will contain the same contents as before, except the curved 
quotes are magically transformed to grave accent and apostrophe on the display. 
  It's weird that copied text mysteriously changes its display representation at 
a seemingly unrelated moment.

How about the following idea instead.  Instead of displaying grave accent and 
apostrophe specially, have with-help-window transliterate these characters in 
place before displaying itself as usual.  with-help-window should not 
transliterate characters that are marked as being escaped, or as being user data 
(not clear that we need two kinds of marks here; one should do, no?).

That's the big picture.  Here are a few more-minor remarks.

> +  (font-lock-add-keywords
> +   nil '(("\\(\\\\~\\)\\(?:\\\\~\\|.\\)"

As already mentioned, the new \~ quoting syntax doesn't seem to be needed; we 
can get by with the existing \= quoting syntax.  So let's go that direction.

Also, this regexp string matches either (1) \~\~ or (2) \~ followed by any 
non-newline character.  But if \~ is supposed to escape the next character, the 
regexp string should simply implement that, i.e., the regexp string should be 
"\\\\~\\(.\\|\n\\)".

I assume the extra complication is about escaping backslash itself, e.g., if the 
docstring is \~\ (which would look like "\\~\\" in the source code) this should 
stand for \ in the *Help* buffer.  But the above regexp doesn't do that. If we 
were to keep \~ perhaps we could do the simple regexp match first, and then look 
backwards programatically just before the matching \~ and verifying that either 
(1) it's not preceded by ordinary ~ or (2) it's preceded by ordinary ~ but the ~ 
is not preceded by ordinary \.  By "ordinary" I mean a character that does not 
have the help-value or help-tilde-escaped properties.

But all and all this is sounding way tooo complicated and we should simply use 
the same escape sequences we've always been using.

> +               (unless (get-text-property mbeg 'help-value)

Supposed the matched string is partly help-value, and partly not.  E.g., mbeg 
has help-value but mbeg+1 does not but mbeg+2 does.  Shouldn't this test that 
all the matched characters are not help-value characters?

> +                 ;; If we use "" as the third argument, cursor
> +                 ;; stumbles once when moving over its position.

I don't understand this comment.  Can you explain?  For example, does the 
comment apply to the just the compose-region call, or to the rest of the 'unless'?

> +                                 (buffer-substring-no-properties
> +                                  mend (1+ mend)))

This may go haywire if it returns "\t", because a TAB is special to 
compose-region.  Also, what if the buffer has some properties other than 
help-value that should be preserved?

At this point I stopped reviewing the changes in detail, as really the big 
picture should be addressed first.