bug#3400: bug in add-text-properties

bug#3400: bug in add-text-properties

[Werner LEMBERG]

[-- Attachment #1: Type: Text/Plain, Size: 1271 bytes --]


[emacs CVS build 2009-05-13]


Steps to repeat the bug:

  0. Start `emacs -Q'.

  1. Load attached file `emacs-bug.txt' with `C-x C-f'.

  2. Load attached file `emacs-bug.el' with `load-file'.

  3. Say `M-x make-first-line-invisible'.

The first line properly disappears, however, the first character in
the now visible part of the buffer (the letter `a') makes problems: It
incorrectly refers to the character at buffer position 1 (the digit
`1').  In particular, `C-u C-x =' for letter `a' shows

---

        character: 1 (49, #o61, #x31)
preferred charset: ascii (ASCII (ISO646 IRV))
       code point: 0x31
           syntax: w 	which means: word
         category: .:Base, a:ASCII, l:Latin, r:Roman
      buffer code: #x31
        file code: #x31 (encoded by coding system undecided-unix)
          display: by this font (glyph code)
    xft:-unknown-DejaVu Sans Mono-normal-normal-normal-*-21-*-*-*-m-0-iso10646-1 (#x14)

Character code properties: customize what to show
  name: DIGIT ONE
  general-category: Nd (Number, Decimal Digit)

There are text properties here:
  intangible           t
  invisible            t

---

A side effect is that `C-a' no longer moves to the beginning of the
line.


    Werner

[-- Attachment #2: emacs-bug.txt --]
[-- Type: Text/Plain, Size: 12 bytes --]

12345
abcde

[-- Attachment #3: emacs-bug.el --]
[-- Type: Text/Plain, Size: 254 bytes --]

(defun make-first-line-invisible ()
  (interactive)
  (goto-char (point-min))
  (setq start (line-beginning-position))
  (setq end (1+ (line-end-position))) ; handle \n
  (add-text-properties start
		       end
		       '(invisible t
			 intangible t)))

bug#3400: bug in add-text-properties

[martin rudalics]

 > The first line properly disappears, however, the first character in
 > the now visible part of the buffer (the letter `a') makes problems: It
 > incorrectly refers to the character at buffer position 1 (the digit
 > `1').  In particular, `C-u C-x =' for letter `a' shows
 >
 > ---
 >
 >         character: 1 (49, #o61, #x31)
[...]
 > There are text properties here:
 >   intangible           t
 >   invisible            t

`add-text-properties' describes the character after point and when you
call `describe-char' point is at the beginning of the buffer.  So IMHO
this is not a bug in `add-text-properties'.  It's arguable whether point
should be allowed to be there in your case, but then where to move point
when the entire buffer is invisible?

 > A side effect is that `C-a' no longer moves to the beginning of the
 > line.

A side effect of what?

martin

bug#3400: bug in add-text-properties

[Werner LEMBERG]

>> The first line properly disappears, however, the first character in
>> the now visible part of the buffer (the letter `a') makes problems:
>> It incorrectly refers to the character at buffer position 1 (the
>> digit `1').  In particular, `C-u C-x =' for letter `a' shows
>>
>> ---
>>
>>         character: 1 (49, #o61, #x31)
> [...]
>> There are text properties here:
>>   intangible           t
>>   invisible            t
>
> `add-text-properties' describes the character after point and when
> you call `describe-char' point is at the beginning of the buffer.

Hmm.  If I have

   12345
   ABCDE

and I make `12345' + the newline after the `5' invisible and
intangible, I see

  ABCDE

in the buffer.  If the blinking cursor stays on the `A', and you press
`C-x =', do you *really* expect to see the properties of character `1'
instead of `A'?  This is completely counterintuitive IMHO.

> So IMHO this is not a bug in `add-text-properties'.

Maybe.  I don't know the internal details of Emacs.  However, calling
add-text-properties triggers the problem I experience.

> It's arguable whether point should be allowed to be there in your
> case,

In this particular case I don't care where `point' is.  I want to get
the correct result of `C-x ='.

> but then where to move point when the entire buffer is invisible?

This might be handled as a special case.

>> A side effect is that `C-a' no longer moves to the beginning of the
>> line.
>
> A side effect of what?

Of calling add-text-properties as described in my report.  I do expect
that C-a moves the blinking cursor to the beginning of the line and
not to the *second* character in the buffer, letter `B'.


    Werner

bug#3400: bug in add-text-properties

[martin rudalics]

 > Hmm.  If I have
 >
 >    12345
 >    ABCDE
 >
 > and I make `12345' + the newline after the `5' invisible and
 > intangible, I see
 >
 >   ABCDE
 >
 > in the buffer.  If the blinking cursor stays on the `A', and you press
 > `C-x =', do you *really* expect to see the properties of character `1'
 > instead of `A'?  This is completely counterintuitive IMHO.

Then the value of `point' at the time you invoke that command is
counterintuitive.

 >> So IMHO this is not a bug in `add-text-properties'.
 >
 > Maybe.  I don't know the internal details of Emacs.  However, calling
 > add-text-properties triggers the problem I experience.
 >
 >> It's arguable whether point should be allowed to be there in your
 >> case,
 >
 > In this particular case I don't care where `point' is.  I want to get
 > the correct result of `C-x ='.

Don't blame `describe-char' here.  It must be able to work on any buffer
position including those preceded and/or followed by invisible text.

 >> but then where to move point when the entire buffer is invisible?
 >
 > This might be handled as a special case.

Maybe.  But then you admit that the problem is with the position of
`point' ;-) Anyway, IIRC the idea was that you should be able put a
visible overlay before invisible text at the beginning of the buffer and
be able to move the cursor there.  I have no idea whether that ever
worked, though.

 >>> A side effect is that `C-a' no longer moves to the beginning of the
 >>> line.
 >> A side effect of what?
 >
 > Of calling add-text-properties as described in my report.  I do expect
 > that C-a moves the blinking cursor to the beginning of the line and
 > not to the *second* character in the buffer, letter `B'.

This _is_ a bug and I doubt it's the only one with respect to moving
point over invisible text.

martin

bug#3400: bug in add-text-properties

[Stefan Monnier]

> Hmm.  If I have

>    12345
>    ABCDE

> and I make `12345' + the newline after the `5' invisible and
> intangible, I see

>   ABCDE

> in the buffer.  If the blinking cursor stays on the `A', and you press
> `C-x =', do you *really* expect to see the properties of character `1'
> instead of `A'?  This is completely counterintuitive IMHO.

If you hit `b', the `b' will be inserted right before "12345", and it
will not be invisible.  BTW, why are you using `intangible'?


        Stefan

bug#3400: bug in add-text-properties

[Werner LEMBERG]

>> Hmm.  If I have
> 
>>    12345
>>    ABCDE
> 
>> and I make `12345' + the newline after the `5' invisible and
>> intangible, I see
> 
>>   ABCDE
> 
>> in the buffer.  If the blinking cursor stays on the `A', and you
>> press `C-x =', do you *really* expect to see the properties of
>> character `1' instead of `A'?  This is completely counterintuitive
>> IMHO.
> 
> If you hit `b', the `b' will be inserted right before "12345", and it
> will not be invisible.

So what?  With `C-x =' I see `A' and get the data of `1', this is what
I consider a bug.  Actually, there is no way to retrieve information
from `A' at all.

> BTW, why are you using `intangible'?

I've just copied some code -- I haven't checked whether I can omit it.


BTW, the problem is not restricted to the beginning of the buffer.
With this input

  12345
  ABCDE
  abcde

and this lisp code

  (defun make-second-line-invisible ()
    (interactive)
    (goto-line 2)
    (setq start (line-beginning-position))
    (setq end (1+ (line-end-position))) ; handle \n
    (add-text-properties start
			 end
			 '(invisible t
			   intangible t)))

there is the same problem: After calling the function you get

  12345
  abcde

and `C-x =' (positioned on the `a') reports the info of the now
invisible character `A'.  Similarly, C-a only moves to character `b'.


    Werner

bug#3400: bug in add-text-properties

[Stefan Monnier]

>> If you hit `b', the `b' will be inserted right before "12345", and it
>> will not be invisible.
> So what?  With `C-x =' I see `A' and get the data of `1',

Point is at position 1 (i.e. just before the char "1" rather than
before the char "A"), so it makes sense for C-h = to return data about
that position rather than about some other position one line down.

Also it makes sense to draw the cursor over the "A" because there's
nowhere else to draw it.  Basically, since the chars "12345\n" are
invisible and intangible, the display should pretend they don't exist,
but C-x = is designed to show you the actual underlying data.

> Actually, there is no way to retrieve information from `A' at all.

Your `intangible' span basically requests "12345\nA" to be treated as one
indivisible entity, so it's indeed difficult to get info about most of
those individual chars.

>> BTW, why are you using `intangible'?
> I've just copied some code -- I haven't checked whether I can omit it.

First things first: if you're not 200% sure you absolutely need
`intangible', then don't use it.  It's a nasty beast.


        Stefan

bug#3400: bug in add-text-properties

[Werner LEMBERG]

> Point is at position 1 (i.e. just before the char "1" rather than
> before the char "A"), so it makes sense for C-h = to return data
> about that position rather than about some other position one line
> down.

I disagree.  Whereever point is, it should report the character after
point which is accessible (this is, visible).  Perhaps it makes sense
to add a red warning message in the output of `C-x =' that point is
not located right before the displayed character.

If we had a small line instead of a blinking, block-like cursor (as it
is used with, say, OpenOffice), I could accept your argumentation.
But the visual effect is overwhelming, and it is extremely confusing
if the optical appearance differs so much from the `theoretical'
result.

>> Actually, there is no way to retrieve information from `A' at all.
> 
> Your `intangible' span basically requests "12345\nA" to be treated
> as one indivisible entity, so it's indeed difficult to get info
> about most of those individual chars.

Hmm.  IMHO, the same argumentation as above holds.

> if you're not 200% sure you absolutely need `intangible', then don't
> use it.  It's a nasty beast.

Indeed.  But for my particular needs -- filtering out some lines, then
searching and editing the visible ones only without accidentally
editing invisible stuff, it seems that `intangible' is still
necessary, and probably will stay so.


    Werner

bug#3400: bug in add-text-properties

[Lennart Borgman]

On Fri, May 29, 2009 at 9:00 AM, Werner LEMBERG <wl@gnu.org> wrote:
>
>> Point is at position 1 (i.e. just before the char "1" rather than
>> before the char "A"), so it makes sense for C-h = to return data
>> about that position rather than about some other position one line
>> down.
>
> I disagree.  Whereever point is, it should report the character after
> point which is accessible (this is, visible).  Perhaps it makes sense
> to add a red warning message in the output of `C-x =' that point is
> not located right before the displayed character.

I agree with you here, Werner. And I think the red warning is a good idea too.

bug#3400: bug in add-text-properties

[Drew Adams]

> >> Point is at position 1 (i.e. just before the char "1" rather than
> >> before the char "A"), so it makes sense for C-h = to return data
> >> about that position rather than about some other position one line
> >> down.
> >
> > I disagree.  Whereever point is, it should report the 
> > character after point which is accessible (this is, visible).
> > Perhaps it makes sense to add a red warning message in the
> > output of `C-x =' that point is
> > not located right before the displayed character.
> 
> I agree with you here, Werner. And I think the red warning is 
> a good idea too.

I haven't followed this thread, so apologies if I miss the point.

To me, we want C-x = to give as much info as possible, including info about
positions that we might not be able to see or even to reach/access by
positioning point.

Since there already is a way to get info about the positions that are accessible
and those that are visible, by moving the cursor a bit, there is no loss in also
providing info about positions that are inaccessible or invisible. So I think I
agree with Stefan here (if this is in fact what he says): C-x = should return
info about the closest position you can access.

That is, if C-x = can help by giving some info about invisible or intangible
chars, so much the better. Nothing prevents us from moving the cursor a bit and
getting the info about the accessible, visible, boundary positions.

If the info shown for such unusual positions is accurate, it will, itself,
indicate that the char is intangible or invisible or whatever. But I do agree
that perhaps some extra notification (red or, preferably just an additional
statement) of this might be helful. (But please don't call such a notification a
"warning".)

Again, ignore if not relevant.

bug#3400: bug in add-text-properties

[Stefan Monnier]

> I disagree.  Whereever point is, it should report the character after
> point which is accessible (this is, visible).  Perhaps it makes sense
> to add a red warning message in the output of `C-x =' that point is
> not located right before the displayed character.

I think what would make most sense if for C-x = to notice such invisible
text and either give info about the whole invisible area, or prompt
which position within that area is actually desired.  In your case, you
want info about the visible char, but I most often use C-x =
specifically to get info about the part I can't see.

> If we had a small line instead of a blinking, block-like cursor (as it
> is used with, say, OpenOffice), I could accept your argumentation.

That can be configured via `cursor-type'.

>> if you're not 200% sure you absolutely need `intangible', then don't
>> use it.  It's a nasty beast.
> Indeed.  But for my particular needs -- filtering out some lines, then
> searching and editing the visible ones only without accidentally
> editing invisible stuff, it seems that `intangible' is still
> necessary, and probably will stay so.

Your call, but you've been warned.  I'd still recommend you first try it
without `intangible'.


        Stefan

bug#3400: bug in add-text-properties

[Werner LEMBERG]

> I think what would make most sense if for C-x = to notice such
> invisible text and either give info about the whole invisible area,
> or prompt which position within that area is actually desired.

Yes, this sounds like a good idea.

What do you think about the C-a issue, BTW?

> In your case, you want info about the visible char, but I most often
> use C-x = specifically to get info about the part I can't see.

Honestly, up to now I had never the idea of using C-x = like that.  I
wasn't even aware that I can use it for that purpose.

>> If we had a small line instead of a blinking, block-like cursor (as
>> it is used with, say, OpenOffice), I could accept your
>> argumentation.
> 
> That can be configured via `cursor-type'.

I know, but the default is a block cursor...


     Werner

bug#3400: bug in add-text-properties

[Stefan Monnier]

>> I think what would make most sense if for C-x = to notice such
>> invisible text and either give info about the whole invisible area,
>> or prompt which position within that area is actually desired.
> Yes, this sounds like a good idea.

Patches welcome ;-)

> What do you think about the C-a issue, BTW?

That would be a bug in C-a, but I can't reproduce it: if I go to "c" and
hit C-a, my cursor correctly moves to the beginning of line (just
before "A", displayed as being just before "a").


        Stefan

bug#3400: bug in add-text-properties

[martin rudalics]

>> What do you think about the C-a issue, BTW?
> 
> That would be a bug in C-a, but I can't reproduce it: if I go to "c" and
> hit C-a, my cursor correctly moves to the beginning of line (just
> before "A", displayed as being just before "a").

It's a Heisenbug.

martin

bug#3400: bug in add-text-properties

[Werner LEMBERG]

>>> What do you think about the C-a issue, BTW?
>> That would be a bug in C-a, but I can't reproduce it: if I go to "c"
>> and
>> hit C-a, my cursor correctly moves to the beginning of line (just
>> before "A", displayed as being just before "a").
> 
> It's a Heisenbug.

Hmm.  It seems to be a problem with the `intangible' property.  If I
remove it, C-a works correctly.  I'll try a newer Emacs version; maybe
this problem has vanished meanwhile.


    Werner

bug#3400: bug in add-text-properties

[Werner LEMBERG]

>> What do you think about the C-a issue, BTW?
> 
> That would be a bug in C-a, but I can't reproduce it: if I go to "c"
> and hit C-a, my cursor correctly moves to the beginning of line
> (just before "A", displayed as being just before "a").

I've just recompiled current CVS, and I get still the erroneous
behaviour, using the files I've sent in my bug report.

Anyone else who sees this?


    Werner

bug#3400: bug in add-text-properties

[Stephen Berman]

On Sat, 30 May 2009 09:01:34 +0200 (CEST) Werner LEMBERG <wl@gnu.org> wrote:

>>> What do you think about the C-a issue, BTW?
>> 
>> That would be a bug in C-a, but I can't reproduce it: if I go to "c"
>> and hit C-a, my cursor correctly moves to the beginning of line
>> (just before "A", displayed as being just before "a").
>
> I've just recompiled current CVS, and I get still the erroneous
> behaviour, using the files I've sent in my bug report.
>
> Anyone else who sees this?
>
>
>     Werner

I also see the buggy C-a behavior with GNU Emacs 23.0.94.1
(i686-pc-linux-gnu, GTK+ Version 2.14.4) of 2009-05-28 on escher 

Steve Berman

bug#3400: bug in add-text-properties

[martin rudalics]

 > Hmm.  It seems to be a problem with the `intangible' property.  If I
 > remove it, C-a works correctly.  I'll try a newer Emacs version; maybe
 > this problem has vanished meanwhile.

It's not necessarily related to the intangible property.  Have a
look at the thread starting with

http://lists.gnu.org/archive/html/emacs-devel/2007-11/msg00601.html

martin

bug#3400: bug in add-text-properties

[Nicolas Petton]

Stephen Berman <stephen.berman@gmx.net> writes:

> I also see the buggy C-a behavior with GNU Emacs 23.0.94.1
> (i686-pc-linux-gnu, GTK+ Version 2.14.4) of 2009-05-28 on escher

I can't reproduce the C-a behavior issue on Emacs built today, commit
c54238d.

The issue with C-u C-x = is still there though.

Cheers,
Nico

bug#3400: bug in add-text-properties

[npostavs]

retitle 3400 Let C-x = show info about visible and invisible text near point
severity 3400 wishlist
quit

Nicolas Petton <nicolas@petton.fr> writes:

> Stephen Berman <stephen.berman@gmx.net> writes:
>
>> I also see the buggy C-a behavior with GNU Emacs 23.0.94.1
>> (i686-pc-linux-gnu, GTK+ Version 2.14.4) of 2009-05-28 on escher
>
> I can't reproduce the C-a behavior issue on Emacs built today, commit
> c54238d.

Same here (25.1-rc1).

>
> The issue with C-u C-x = is still there though.

So I'm setting this to wishlist.

http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3400#55 is the clearest
statement of what's wished for.

bug#3400: Let C-x = show info about visible and invisible text near point

[Lars Ingebrigtsen]

npostavs@users.sourceforge.net writes:

> retitle 3400 Let C-x = show info about visible and invisible text near point

I've now made the command say something more about invisibility/point
confusion in Emacs 29.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no