Re: Escaping quotes in docstrings, Was: A simple solution to "Upcoming loss of usability ..."

[Dmitry Gutov <dgutov@yandex.ru>]

On 07/01/2015 05:56 AM, Paul Eggert wrote:

> I tried it out.  One thing I noticed right away is odd behavior that
> results from storing grave accent and apostrophe in the buffer and
> displaying them as curved quotes.  For example, run 'emacs -Q -nw', then
> read the documentation for the 'length' function and copy it to a new
> file /tmp/foo by typing "C-h f length RET C-x o C-x h M-w C-x 4 f
> /tmp/foo RET C-y C-s".  On the screen you'll see a buffer 'foo'
> containing curved quotes.  Now, revisit /tmp/foo by typing "C-x C-v
> RET".  The buffer will contain the same contents as before, except the
> curved quotes are magically transformed to grave accent and apostrophe
> on the display.  It's weird that copied text mysteriously changes its
> display representation at a seemingly unrelated moment.

It's the same if you copy some syntax-highlighted text (say, from 
src/doc.c) to /tmp/foo and save it. Kill the buffer, reopen - the 
highlighting is gone! :)

That may seem counter-intuitive, but it's something Emacs users are 
generally familiar with.

> How about the following idea instead.  Instead of displaying grave
> accent and apostrophe specially, have with-help-window transliterate
> these characters in place before displaying itself as usual.

That should work, too. In help-mode-finish, before help-make-xrefs.

> with-help-window should not transliterate characters that are marked as
> being escaped, or as being user data (not clear that we need two kinds
> of marks here; one should do, no?).

They're semantically different. I think we currently apply linkification 
(help-make-xrefs) to the contents of user data, and we might continue 
doing that.

On the other hand, we should probably avoid linkifying symbols inside 
quotes when at least one of the quotes is escaped.

> That's the big picture.  Here are a few more-minor remarks.
>
>> +  (font-lock-add-keywords
>> +   nil '(("\\(\\\\~\\)\\(?:\\\\~\\|.\\)"
>
> As already mentioned, the new \~ quoting syntax doesn't seem to be
> needed; we can get by with the existing \= quoting syntax.  So let's go
> that direction.

For us to go there, could you please make substitute-command-keys add 
the `escaped' property to the escaped characters in its output? And push 
it to scratch/quote-escaping.

> Also, this regexp string matches either (1) \~\~ or (2) \~ followed by
> any non-newline character.  But if \~ is supposed to escape the next
> character, the regexp string should simply implement that, i.e., the
> regexp string should be "\\\\~\\(.\\|\n\\)".

I guess so. Sorry, brain fart.

> I assume the extra complication is about escaping backslash itself,
> e.g., if the docstring is \~\ (which would look like "\\~\\" in the
> source code) this should stand for \ in the *Help* buffer.  But the
> above regexp doesn't do that.

Doesn't it? Works for me. But anyway, let's try to reuse the quoting 
from substitute-command-keys first.

>> +               (unless (get-text-property mbeg 'help-value)
>
> Supposed the matched string is partly help-value, and partly not.  E.g.,
> mbeg has help-value but mbeg+1 does not but mbeg+2 does.  Shouldn't this
> test that all the matched characters are not help-value characters?

Why? I'm assuming the value is separated from the other contents by 
whitespace or newlines. IMHO, that would too defensive. We're throwing 
that code out anyway.

On the other hand, we could encounter a value between ` and ', in 
help--translate-quotes, if the value is shown at the end of the buffer.

Should be taken care of in f9f3aa5 (as well as another nearby bug).

>> +                 ;; If we use "" as the third argument, cursor
>> +                 ;; stumbles once when moving over its position.
>
> I don't understand this comment.  Can you explain?  For example, does
> the comment apply to the just the compose-region call, or to the rest of
> the 'unless'?

Only to the compose-region call. If we pass "" to it as COMPOSITION, the 
result will still be considered a valid position for the cursor, even 
though it has zero width. Anyway, we're going a different route: no 
compose-region calls.

>> +                                 (buffer-substring-no-properties
>> +                                  mend (1+ mend)))
>
> This may go haywire if it returns "\t", because a TAB is special to
> compose-region.  Also, what if the buffer has some properties other than
> help-value that should be preserved?

Err, I don't think the current code deletes any existing properties, it 
only changes how the buffer looks.

On the other hand, if help-mode-finish performs the translations 
(destructively, I'm assuming), then we indeed might lose some properties.

I wouldn't worry too much about that, though. If help-mode doesn't know 
about them, no other code is likely to use them either.