M-w, then C-y. C-y inserts text properties that aren't on original.

M-w, then C-y. C-y inserts text properties that aren't on original.

[Alan Mackenzie]

In a C++ Mode buffer, I create a region containing a template-opener <,
M-w it, then C-y it.

The C-y puts a syntax-table property on the (copy of) the <, but this
property didn't exist on the original.

What _was_ on the original was a category property whose symbol had a
syntax-table property.

This is the root cause of bug 5570 (indentation messed up in C++).

Would somebody please suggest the part of Emacs I should be looking at
to debug this problem.

Thanks!

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: M-w, then C-y. C-y inserts text properties that aren't on original.

[Stefan Monnier]

> In a C++ Mode buffer, I create a region containing a template-opener <,
> M-w it, then C-y it.
> The C-y puts a syntax-table property on the (copy of) the <, but this
> property didn't exist on the original.
[...]
> Would somebody please suggest the part of Emacs I should be looking at
> to debug this problem.

Can't think of anything (other than the font-lock rules which are under
c++-mode's control, of course).


        Stefan

Re: M-w, then C-y. C-y inserts text properties that aren't on original.

[Eli Zaretskii]

> Date: Thu, 18 Feb 2010 20:29:02 +0000
> From: Alan Mackenzie <acm@muc.de>
> 
> In a C++ Mode buffer, I create a region containing a template-opener <,
> M-w it, then C-y it.
> 
> The C-y puts a syntax-table property on the (copy of) the <, but this
> property didn't exist on the original.
> 
> What _was_ on the original was a category property whose symbol had a
> syntax-table property.
> 
> This is the root cause of bug 5570 (indentation messed up in C++).
> 
> Would somebody please suggest the part of Emacs I should be looking at
> to debug this problem.

Can you please give a precise recipe, starting with "emacs -Q", for
reproducing the problem?  I don't know enough about C++ Mode internals
to understand what you say.

Thanks.

Re: M-w, then C-y. C-y inserts text properties that aren't on original.

[Alan Mackenzie]

Hi, Eli,

On Fri, Feb 19, 2010 at 11:22:55AM +0200, Eli Zaretskii wrote:
> > Date: Thu, 18 Feb 2010 20:29:02 +0000
> > From: Alan Mackenzie <acm@muc.de>

> > In a C++ Mode buffer, I create a region containing a template-opener <,
> > M-w it, then C-y it.

> > The C-y puts a syntax-table property on the (copy of) the <, but this
> > property didn't exist on the original.

> > What _was_ on the original was a category property whose symbol had a
> > syntax-table property.

> > This is the root cause of bug 5570 (indentation messed up in C++).

> > Would somebody please suggest the part of Emacs I should be looking at
> > to debug this problem.

> Can you please give a precise recipe, starting with "emacs -Q", for
> reproducing the problem?  I don't know enough about C++ Mode internals
> to understand what you say.

#########################################################################
emacs -Q
(defvar propp-var nil)
(put 'propp-var 'foo t)

Create buffer "foo" and put a line of text (including NL) into it.

(with-current-buffer "foo" (put-text-property 1 2 'category 'propp-var))

In foo:
M-<                                     ; goto BOB
C-<space> C-n M-w                       ; Copy L1 into the kill ring
C-y                                     ; Yank it as L2

M-<   C-u C-x =                         ; Get details of char at BOB
    There are text properties here:
      category             propp-var    ; <==============================
      fontified            t

C-n   C-u C-x =                         ; Get details of char at BOL 2
    There are text properties here:
      fontified            t
      foo                  t            ; <==============================
#########################################################################

Note that where the original "had" a 'foo property by indirection through
the 'category, the copy has spuriously short-circuited the 'category
indirection.

> Thanks.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: M-w, then C-y. C-y inserts text properties that aren't on original.

[Alan Mackenzie]

On Fri, Feb 19, 2010 at 12:24:19PM +0000, Alan Mackenzie wrote:

> On Fri, Feb 19, 2010 at 11:22:55AM +0200, Eli Zaretskii wrote:

> > Can you please give a precise recipe, starting with "emacs -Q", for
> > reproducing the problem?  I don't know enough about C++ Mode internals
> > to understand what you say.

> #########################################################################
> emacs -Q
> (defvar propp-var nil)
> (put 'propp-var 'foo t)

> Create buffer "foo" and put a line of text (including NL) into it.

> (with-current-buffer "foo" (put-text-property 1 2 'category 'propp-var))

> In foo:
> M-<                                     ; goto BOB
> C-<space> C-n M-w                       ; Copy L1 into the kill ring
> C-y                                     ; Yank it as L2

> M-<   C-u C-x =                         ; Get details of char at BOB
>     There are text properties here:
>       category             propp-var    ; <==============================
>       fontified            t

> C-n   C-u C-x =                         ; Get details of char at BOL 2
>     There are text properties here:
>       fontified            t
>       foo                  t            ; <==============================
> #########################################################################

> Note that where the original "had" a 'foo property by indirection through
> the 'category, the copy has spuriously short-circuited the 'category
> indirection.

I've found it.

`yank' calls `insert-for-yank' calls `insert', which inserts the top of
the kill ring correctly into the buffer.  Then...

`insert-for-yank' carries on, and calls `remove-yank-excluded-properties'
which deliberately changes 'category properties into other properties.

Presumably there was some special case in the distant past which required
this transformation.  It is surely wrong in the general case - yanking
into a buffer shouldn't add text properties which weren't in the
original - surely?

I'm going to try and track down that special case.  It looks like I'll
have to fix the inserted string in CC Mode by hand.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: M-w, then C-y. C-y inserts text properties that aren't on original.

[Eli Zaretskii]

> Date: Fri, 19 Feb 2010 14:35:13 +0000
> Cc: emacs-devel@gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> `insert-for-yank' carries on, and calls `remove-yank-excluded-properties'
> which deliberately changes 'category properties into other properties.
> 
> Presumably there was some special case in the distant past which required
> this transformation.  It is surely wrong in the general case - yanking
> into a buffer shouldn't add text properties which weren't in the
> original - surely?
> 
> I'm going to try and track down that special case.  It looks like I'll
> have to fix the inserted string in CC Mode by hand.

The reason for this behavior is this change:

    2002-04-27  Richard M. Stallman  <rms@gnu.org>

	    * subr.el (insert-for-yank): Replace `category' property
	    with whatever properties it stands for.

The discussion that led to this change starts here:

  http://lists.gnu.org/archive/html/emacs-devel/2002-04/msg00648.html

You will find there the reason for the change (buttons in *Help*
buffers, and the way we put the mouse-face property on them).

I wonder if we could make the code which replaces `category' with what
it stands for, with a slightly less general code that replaces only
those values of `category' that are used for buttons.  Or some similar
change which will make that feature more focused on the problem it
needs to solve.

I added a comment in `remove-yank-excluded-properties' explaining what
do we do that for and pointing to the above URL for details.

Re: M-w, then C-y. C-y inserts text properties that aren't on original.

[Eli Zaretskii]

> Date: Sat, 20 Feb 2010 11:02:39 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: emacs-devel@gnu.org
> 
> I wonder if we could make the code which replaces `category' with what
> it stands for, with a slightly less general code that replaces only
> those values of `category' that are used for buttons.  Or some similar
> change which will make that feature more focused on the problem it
> needs to solve.

Here's one idea: make a deep search through the properties of the
value of the `category' property, looking for any of the properties in
`yank-excluded-properties'.  If found, do the replacement we do now.

Re: M-w, then C-y. C-y inserts text properties that aren't on original.

[Eli Zaretskii]

> Date: Sun, 21 Feb 2010 21:59:51 +0000
> Cc: emacs-devel@gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> > Here's one idea: make a deep search through the properties of the
> > value of the `category' property, looking for any of the properties in
> > `yank-excluded-properties'.  If found, do the replacement we do now.
> 
> I'm not in favour of anything like this.
> 
> The way the current replacement works, it replaces category properties
> with "hard" properties.  These "hard" properties then foul up future
> category properties, should you apply a different category, or change
> the properties on the category symbol.
> 
> As Richard said back in 2002, it's a difficult problem to fix, because
> there's no general or canonical case.  I think a better way would have
> been simply to drop category properties.  But we've got what we've got,
> so why not just stick with it?

I thought you wanted to fix the original problem, didn't you?  So I
suggested one way of having it fixed.  If we leave things as they are,
the problem you are having will stay as well, no?

Re: M-w, then C-y. C-y inserts text properties that aren't on original.

[Alan Mackenzie]

Hi, Eli,

On Mon, Feb 22, 2010 at 06:12:12AM +0200, Eli Zaretskii wrote:
> > Date: Sun, 21 Feb 2010 21:59:51 +0000
> > Cc: emacs-devel@gnu.org
> > From: Alan Mackenzie <acm@muc.de>

> > > Here's one idea: make a deep search through the properties of the
> > > value of the `category' property, looking for any of the properties
> > > in `yank-excluded-properties'.  If found, do the replacement we do
> > > now.

> > I'm not in favour of anything like this.

> > The way the current replacement works, it replaces category
> > properties with "hard" properties.  These "hard" properties then foul
> > up future category properties, should you apply a different category,
> > or change the properties on the category symbol.

> > As Richard said back in 2002, it's a difficult problem to fix,
> > because there's no general or canonical case.  I think a better way
> > would have been simply to drop category properties.  But we've got
> > what we've got, so why not just stick with it?

> I thought you wanted to fix the original problem, didn't you?  So I
> suggested one way of having it fixed.  If we leave things as they are,
> the problem you are having will stay as well, no?

I was wanting both to fix "my" problem and fix the ostensible bug which
was messing up the text properties.

I've decided to fix the CC Mode problem by ad hoc means, rather than try
to push through a quite big change to TP handling.  It means that certain
characters in a C++ Mode buffer will get text properties of
(risky-local-variable t), but that won't harm anything and will serve as
a reminder of the ridiculousness of life, the universe, and everything.

Do you think that the idea you suggest (above) would be better than the
current way?  I dislike it because it looks more complicated, and I think
it would be as least as difficult to code round as the current way.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: M-w, then C-y. C-y inserts text properties that aren't on original.

[MON KEY]

,----
| The root cause of this mess is the confusion in text properties.
| Text properties (as opposed to overlays) become part of the text
| they're applied to, yet they're mostly used (e.g. for font locking)
| in situations where they're not intrinsically part of the text.
`----

The ``moslty'' generalization is far too broad a stroke.
You seem to be reducing this to a matter of presentation.

,----
| It would be better if these uses could use overlays, then copying
| the string would ignore the overlay.
`----

No, it would not.

There are a good many uses (some as yet poorly explored) where text
properties are the more suitable ``container'' than the equivalent
approach with overlays.

If anything it would be better if overlays were further deprecated.

/s_P\

Re: M-w, then C-y. C-y inserts text properties that aren't on original.

[Daniel Colascione]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi, Alan.

On 2/22/10 7:56 AM, Alan Mackenzie wrote:
> On Mon, Feb 22, 2010 at 06:12:12AM +0200, Eli Zaretskii wrote:
>>> Date: Sun, 21 Feb 2010 21:59:51 +0000
>>> Cc: emacs-devel@gnu.org
>>> From: Alan Mackenzie <acm@muc.de>
> 
>>>> Here's one idea: make a deep search through the properties of the
>>>> value of the `category' property, looking for any of the properties
>>>> in `yank-excluded-properties'.  If found, do the replacement we do
>>>> now.
> 
> I've decided to fix the CC Mode problem by ad hoc means, rather than try
> to push through a quite big change to TP handling.

It also means that the next person who wants to use category properties
will be bitten by exactly the same lurking gotcha. It'd be better to
employ the yank-excluded-properties approach and fix the problem once
and for all. It seems reasonable and somewhat symmetric.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (Darwin)

iEYEARECAAYFAkuDAQEACgkQ17c2LVA10VsrTwCeOYbsxkVy80AebVMVyx0QQADS
FrUAoKqoslz0ncAENYoJFi5ybiOXYbR5
=rF//
-----END PGP SIGNATURE-----

Re: M-w, then C-y. C-y inserts text properties that aren't on original.

[Stefan Monnier]

>> I wonder if we could make the code which replaces `category' with what
>> it stands for, with a slightly less general code that replaces only
>> those values of `category' that are used for buttons.  Or some similar
>> change which will make that feature more focused on the problem it
>> needs to solve.

> Here's one idea: make a deep search through the properties of the
> value of the `category' property, looking for any of the properties in
> `yank-excluded-properties'.  If found, do the replacement we do now.

Another one is to use of a `yank-excluded-category' property on the
symbols used for categories to decide how to handle them.


        Stefan