bug#3312: 23.0.93; Use punctuation syntax for apostrophe in Info

bug#3312: 23.0.93; Use punctuation syntax for apostrophe in Info

[Drew Adams]

I don't know if this would be appropriate for all Info manuals, but at
least for Emacs, Elisp, Eintr, and Efaq it makes sense, I think.
 
Apostrophe, or single quote, (') currently has word-constituent syntax
in Info, which it inherits from text-mode.
 
This means that if point is inside some `...'  string, and you use
`C-s C-w', you will pick up also the ' at the end as part of the
search string, which is typically not what you want.
 
If the syntax class for ' were `punctuation' instead of `word', then
you could more easily pick up key sequences, function names, and
variable names that are quoted: `C-x C-x' or `char-syntax', for
example.
 
Yes, you can today pick up everything, including the ` and the ', by
positioning the cursor on the `, but if you want just the quoted
content or some suffix of it, and not the quoting delimiters, you
get the ' too, which is not what you want.
 
If this makes sense for all Info manuals, then a simple fix would
presumably be to change this (in the definition of `info-mode'):
 
(set-syntax-table text-mode-syntax-table)
 
To this:
 
(set-syntax-table
  (copy-syntax-table text-mode-syntax-table))
(modify-syntax-entry ?' ".")
 
I don't know much about text-mode, but I wonder if even there it might
make sense for ' to have syntax class `punctuation' instead of
`word'. Probably the answer is "no", or that would already be the
case. ;-)
 
I see this in `text-mode.el':
 ;; We add `p' so that M-c on 'hello' leads to 'Hello'
 ;; rather than 'hello'.
 (modify-syntax-entry ?' "w p" st)
 
That explains why p was added, but not why ' should have syntax class
w at all.  Again, there probably is a good reason, but I don't know
what it is.  After changing the syntax to punctuation, M-c still seems
to act as desired on 'hello'.
 
In GNU Emacs 23.0.93.1 (i386-mingw-nt5.1.2600)
 of 2009-05-02 on SOFT-MJASON
Windowing system distributor `Microsoft Corp.', version 5.1.2600
configured using `configure --with-gcc (3.4)'
 

bug#3312: 23.0.93; Use punctuation syntax for apostrophe in Info

[Kevin Rodgers]

Drew Adams wrote:
> I don't know much about text-mode, but I wonder if even there it might
> make sense for ' to have syntax class `punctuation' instead of
> `word'. Probably the answer is "no", or that would already be the
> case. ;-)
>  
> I see this in `text-mode.el':
>  ;; We add `p' so that M-c on 'hello' leads to 'Hello'
>  ;; rather than 'hello'.
>  (modify-syntax-entry ?' "w p" st)
>  
> That explains why p was added, but not why ' should have syntax class
> w at all.  Again, there probably is a good reason, but I don't know
> what it is.  After changing the syntax to punctuation, M-c still seems
> to act as desired on 'hello'.

FWIW, I've got this in my ~/.emacs:

(add-hook 'text-mode-hook
           (defun text-mode-punctuation-syntax ()
             ;; Change non-letter characters from word syntax to 
punctuation:
             (modify-syntax-entry ?\240 ".") ; NO-BREAK SPACE
             (modify-syntax-entry ?' ".")))  ; APOSTROPHE

-- 
Kevin Rodgers
Denver, Colorado, USA

bug#3312: 23.0.93; Use punctuation syntax for apostrophe in Info

[Drew Adams]

> > I don't know much about text-mode, but I wonder if even 
> > there it might make sense for ' to have syntax class
> > `punctuation' instead of `word'. Probably the answer
> > is "no", or that would already be the case. ;-)
> >  
> > I see this in `text-mode.el':
> >  ;; We add `p' so that M-c on 'hello' leads to 'Hello'
> >  ;; rather than 'hello'.
> >  (modify-syntax-entry ?' "w p" st)
> >  
> > That explains why p was added, but not why ' should have 
> > syntax class w at all.  Again, there probably is a good
> > reason, but I don't know what it is.  After changing the
> > syntax to punctuation, M-c still seems to act as desired
> > on 'hello'.
> 
> FWIW, I've got this in my ~/.emacs:
> 
> (add-hook 'text-mode-hook
>            (defun text-mode-punctuation-syntax ()
> ;; Change non-letter characters from word syntax to punctuation:
>              (modify-syntax-entry ?\240 ".") ; NO-BREAK SPACE
>              (modify-syntax-entry ?' ".")))  ; APOSTROPHE

Yes, but I wasn't sure that the changes should also be made to text-mode itself
(dunno). For now, I was thinking of having info-mode do `copy-syntax-table' and
then modify that copy.

bug#3312: 23.0.93; Use punctuation syntax for apostrophe in Info

[Juri Linkov]

> If this makes sense for all Info manuals, then a simple fix would
> presumably be to change this (in the definition of `info-mode'):
>
> (set-syntax-table text-mode-syntax-table)
>
> To this:
>
> (set-syntax-table
>   (copy-syntax-table text-mode-syntax-table))
> (modify-syntax-entry ?' ".")

From my .emacs:

     (add-hook 'Info-mode-hook
               (lambda ()
                 (let ((st (copy-syntax-table text-mode-syntax-table)))
                   (modify-syntax-entry ?' "." st)
                   (set-syntax-table st))))

So this is a real problem for Info manuals.

> I don't know much about text-mode, but I wonder if even there it might
> make sense for ' to have syntax class `punctuation' instead of
> `word'. Probably the answer is "no", or that would already be the
> case. ;-)

AFAIK, the apostrophe has word syntax in text-mode to allow treating
word contractions as one word (can't, it's, I'll).  I don't know
whether this is grammatically correct or not but at least the Unicode says
that the category of the apostrophe is Punctuation, so perhaps we should
change apostrophe syntax to punctuation in text-mode as well.

-- 
Juri Linkov
http://www.jurta.org/emacs/

bug#3312: 23.0.93; Use punctuation syntax for apostrophe in Info

[Eli Zaretskii]

> From: Juri Linkov <juri@jurta.org>
> Date: Sun, 17 May 2009 21:59:59 +0300
> Cc: emacs-pretest-bug@gnu.org, 3312@emacsbugs.donarmstrong.com
> 
> I don't know whether this is grammatically correct or not but at
> least the Unicode says that the category of the apostrophe is
> Punctuation, so perhaps we should change apostrophe syntax to
> punctuation in text-mode as well.

Unicode and Emacs have different goals, so we should look for good
reasons that stem from Emacs usage, not follow external standards
blindly.

bug#3312: 23.0.93; Use punctuation syntax for apostrophe in Info

[Drew Adams]

> So this is a real problem for Info manuals.
> 
> > I don't know much about text-mode, but I wonder if even 
> > there it might make sense for ' to have syntax class
> > `punctuation' instead of `word'. Probably the answer is
> > "no", or that would already be the case. ;-)
> 
> AFAIK, the apostrophe has word syntax in text-mode to allow treating
> word contractions as one word (can't, it's, I'll).  I don't know
> whether this is grammatically correct or not

Yes, that is correct, and it is a good reason, as far as it goes.

The comment in the code is way off target, it seems.
What should be said there is what you said here: we treat apostrophe as
word-constituent because of its use in contractions.

Similarly, for possessive forms: the dog's bone.

But in both cases, contractions and possessives, it is a choice whether we want
to treat the apostrophe as word-constituent. The argument in favor is stronger
in the case of contractions - for the possessive "dog's" it is not clear that
you would want to include the "'s" when selecting a word.

> but at least the Unicode says that the category of the
> apostrophe is Punctuation, so perhaps we should
> change apostrophe syntax to punctuation in text-mode as well.

It _is_ punctuation, even in contractions such as "can't". The problem is that
it is punctuation _within a word_, so Emacs must choose whether it wants to
treat it as part of the word (for convenience) or strictly as punctuation.

I would probably vote for treating the syntax class in text mode as punctuation,
not word, but it would be good to hear other points of view.

I do feel that within Info mode it should be treated as punctuation. The use of
`...' forms in Info (even beyond Emacs doc) far outweighs the use of
contractions. In technical doc, we generally avoid using contractions anyway.

bug#3312: 23.0.93; Use punctuation syntax for apostrophe in Info

[Stefan Monnier]

> Apostrophe, or single quote, (') currently has word-constituent syntax
> in Info, which it inherits from text-mode.
 
> This means that if point is inside some `...'  string, and you use
> `C-s C-w', you will pick up also the ' at the end as part of the
> search string, which is typically not what you want.

Yes, it might be a good idea to change it in Info-mode, which is quite
different from text-mode because of the extensive use of `...' and the
fact that it's read-only.
 
> I don't know much about text-mode, but I wonder if even there it might
> make sense for ' to have syntax class `punctuation' instead of
> `word'. Probably the answer is "no", or that would already be the
> case. ;-)
 
Indeed, the answer is no because we want M-u M-u to turn "it's time"
into "It's Time" and not "It'S time".


        Stefan

bug#3312: 23.0.93; Use punctuation syntax for apostrophe in Info

[Drew Adams]

> > I don't know much about text-mode, but I wonder if even 
> > there it might make sense for ' to have syntax class
> > `punctuation' instead of `word'. Probably the answer is
> > "no", or that would already be the case. ;-)
>  
> Indeed, the answer is no because we want M-u M-u to turn "it's time"
> into "It's Time" and not "It'S time".

I think you meant `M-c', but the same reasoning holds for `M-u'.

That's a reason, but it's not necessarily a strong reason. Interactively, you
can as easily do `M-c M-f M-c' to skip over the "s" in "it's".

However, using `capitalize-region', `upcase-region', etc. is another story.
There, your argument is stronger. Still, I personally wonder if it's worth it.
Dunno - I don't know much about how people use text-mode.

bug#3312: 23.0.93; Use punctuation syntax for apostrophe in Info

[Juri Linkov]

>> Apostrophe, or single quote, (') currently has word-constituent syntax
>> in Info, which it inherits from text-mode.
>
>> This means that if point is inside some `...'  string, and you use
>> `C-s C-w', you will pick up also the ' at the end as part of the
>> search string, which is typically not what you want.
>
> Yes, it might be a good idea to change it in Info-mode, which is quite
> different from text-mode because of the extensive use of `...' and the
> fact that it's read-only.

This can be fixed with the following patch (don't know whether now
is the right time to install it):

Index: lisp/info.el
===================================================================
RCS file: /sources/emacs/emacs/lisp/info.el,v
retrieving revision 1.558
diff -u -r1.558 info.el
--- lisp/info.el	14 Jan 2009 23:09:45 -0000	1.558
+++ lisp/info.el	19 May 2009 00:45:36 -0000
@@ -3441,6 +3441,14 @@
 (defvar tool-bar-map)
 (defvar bookmark-make-record-function)
 
+(defvar Info-mode-syntax-table
+  (let ((st (copy-syntax-table text-mode-syntax-table)))
+    ;; Use punctuation syntax for apostrophe because of
+    ;; extensive use of quotes in Info manuals.
+    (modify-syntax-entry ?' "." st)
+    st)
+  "Syntax table used while in `Info-mode'.")
+
 ;; Autoload cookie needed by desktop.el
 ;;;###autoload
 (defun Info-mode ()
@@ -3512,7 +3520,7 @@
   (setq tab-width 8)
   (use-local-map Info-mode-map)
   (add-hook 'activate-menubar-hook 'Info-menu-update nil t)
-  (set-syntax-table text-mode-syntax-table)
+  (set-syntax-table Info-mode-syntax-table)
   (setq local-abbrev-table text-mode-abbrev-table)
   (setq case-fold-search t)
   (setq buffer-read-only t)

-- 
Juri Linkov
http://www.jurta.org/emacs/

bug#3312: 23.0.93; Use punctuation syntax for apostrophe in Info

[Drew Adams]

> From: Juri Linkov Sent: Monday, May 18, 2009 5:48 PM
> >> Apostrophe, or single quote, (') currently has 
> >> word-constituent syntax in Info, which it inherits from text-mode.
> >>
> >> This means that if point is inside some `...'  string, and you use
> >> `C-s C-w', you will pick up also the ' at the end as part of the
> >> search string, which is typically not what you want.
> >
> > Yes, it might be a good idea to change it in Info-mode, 
> > which is quite different from text-mode because of the
> > extensive use of `...' and the fact that it's read-only.
> 
> This can be fixed with the following patch...

That's the same thing I sent in my original bug report:
(set-syntax-table
  (copy-syntax-table text-mode-syntax-table))
(modify-syntax-entry ?' ".")

except that you added a defvar for the table - not needed, but why not? So it's
OK by me.

Dunno whether Kevin's suggestion of doing the same for non-breaking space should
be adopted also: (modify-syntax-entry ?\240 ".")

bug#3312: 23.0.93; Use punctuation syntax for apostrophe in Info

[Drew Adams]

Any chance Juri's patch could be applied now? See also my previous question: do
we want to also treat no-break space the same way?

bug#3312: 23.0.93; Use punctuation syntax for apostrophe in Info

[Lars Magne Ingebrigtsen]

Juri Linkov <juri@jurta.org> writes:

> +    (modify-syntax-entry ?' "." st)

There wasn't much discussion after this, but it seems to me that having
' be 

    (modify-syntax-entry ?\' "w" st)

which it currently is, isn't all that nice in Info buffers.  `M-.' and
the like doesn't work as well as one would like if stuff like `this'
gets treated like "this'" by `M-.'.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

bug#3312: 23.0.93; Use punctuation syntax for apostrophe in Info

[Juri Linkov]

>> +    (modify-syntax-entry ?' "." st)
>
> There wasn't much discussion after this, but it seems to me that having
> ' be
>
>     (modify-syntax-entry ?\' "w" st)
>
> which it currently is, isn't all that nice in Info buffers.  `M-.' and
> the like doesn't work as well as one would like if stuff like `this'
> gets treated like "this'" by `M-.'.

I see that Gnus solves this problem by modifying syntax of ' and `
to whitespace:

  (defvar gnus-article-mode-syntax-table
    (let ((table (copy-syntax-table text-mode-syntax-table)))
      ;; This causes the citation match run O(2^n).
      ;; (modify-syntax-entry ?- "w" table)
      (modify-syntax-entry ?> ")<" table)
      (modify-syntax-entry ?< "(>" table)
      ;; make M-. in article buffers work for `foo' strings
      (modify-syntax-entry ?' " " table)
      (modify-syntax-entry ?` " " table)
      table)
    "Syntax table used in article mode buffers.
  Initialized from `text-mode-syntax-table.")

So I'm not sure what is better for Info buffers:
whitespace syntax (" ") or punctuation (".")?

bug#3312: 23.0.93; Use punctuation syntax for apostrophe in Info

[Drew Adams]

> I see that Gnus solves this problem by modifying syntax of ' and `
> to whitespace...
> So I'm not sure what is better for Info buffers:
> whitespace syntax (" ") or punctuation (".")?

Either is probably OK.  I would vote for punctuation syntax for ` and ',
especially since they are used not only for `...' but also in Lisp code, and
since they are, in English, punctuation.

[We do have embedded spaces in `...' in the manuals (e.g., search for regexp
`\([a-zA-Z]+ \)+[a-zA-Z]+' in (elisp)).  But either whitespace or punctuation
syntax for `' would be OK for that, since you could in either case get all of
the text easily by clicking the mouse (e.g. double-click mouse-1, then click
mouse-3).]

bug#3312: 23.0.93; Use punctuation syntax for apostrophe in Info

[Lars Magne Ingebrigtsen]

Juri Linkov <juri@jurta.org> writes:

> So I'm not sure what is better for Info buffers:
> whitespace syntax (" ") or punctuation (".")?

It probably doesn't matter much, but punctuation sounds more correct.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

bug#3312: 23.0.93; Use punctuation syntax for apostrophe in Info

[Juri Linkov]

> Yes, it might be a good idea to change it in Info-mode, which is quite
> different from text-mode because of the extensive use of `...' and the
> fact that it's read-only.

Fixed and closed.