Info mutilates user overlays.

Info mutilates user overlays.

[Luc Teirlinck]

Is there a reason for the behavior of Info, described below?  It
prevents the user from using his own overlays, as well as from using
any Emacs feature that uses overlays, in Info buffers.  They all get
mysteriously moved and made empty after minimal browsing.  I find it
pretty annoying and unexpected.

emacs-21.3.50 -q --eval "(blink-cursor-mode 0)" &

C-h i

M-: (point-min) Result: 188

M-: (setq ov (make-overlay 190 210)) 
Result: #<overlay from 190 to 210 in *info*>

M-: ov 
Same result (of course).

m Emacs RET d

M-: ov
Result: #<overlay from 1 to 1 in *info*> (Not so "of course" to me.)

Sincerely,

Luc.

Re: Info mutilates user overlays.

[Miles Bader]

Luc Teirlinck <teirllm@dms.auburn.edu> writes:
> Is there a reason for the behavior of Info, described below?  It
> prevents the user from using his own overlays, as well as from using
> any Emacs feature that uses overlays, in Info buffers.

Um, is user overlays in an *info* buffer considered an important
feature?!?  My general impression is that you can't rely on such things
in `special' buffers.

-miles
-- 
.Numeric stability is probably not all that important when you're guessing.

Re: Info mutilates user overlays.

[Luc Teirlinck]

Miles Bader wrote:

   Um, is user overlays in an *info* buffer considered an important
   feature?!?  My general impression is that you can't rely on such things
   in `special' buffers.

That would mean that no Emacs package could use overlays, except in
buffers it manages itself.  At the very least that should be pointed
out in the Elisp section on overlays.  That would really tremendously
limit the usefulness of overlays.  Is there a reason for this?  Why
does Info, or why would any other special buffer, _need_ to do this?

Sincerely,

Luc.

Re: Info mutilates user overlays.

[Luc Teirlinck]

>From my own previous reply:

   Is there a reason for this?  Why does Info, or why would any other
   special buffer, _need_ to do this?

OK I believe I understand now.  It believe it is because the Info
buffer keeps visiting _different_ files during its lifetime.  I
nevertheless believe this should be pointed out in the Elisp manual in
the section on overlays, unless we could think of some way to get
around the problem.

Sincerely,

Luc.

Re: Info mutilates user overlays.

[Miles Bader]

Luc Teirlinck <teirllm@dms.auburn.edu> writes:
> OK I believe I understand now.  It believe it is because the Info
> buffer keeps visiting _different_ files during its lifetime.

So it's pretty normal behavior.

> I nevertheless believe this should be pointed out in the Elisp manual
> in the section on overlays, unless we could think of some way to get
> around the problem.

Can you say why you wanted to use overlays in this case?

-Miles
-- 
"1971 pickup truck; will trade for guns"

Re: Info mutilates user overlays.

[Stefan Monnier]

> emacs-21.3.50 -q --eval "(blink-cursor-mode 0)" &

> C-h i

> M-: (point-min) Result: 188

> M-: (setq ov (make-overlay 190 210)) 
> Result: #<overlay from 190 to 210 in *info*>

> M-: ov 
> Same result (of course).

> m Emacs RET d

> M-: ov
> Result: #<overlay from 1 to 1 in *info*> (Not so "of course" to me.)

You say this is unexpected, but you don't say what you expected.
The text of the buffer has changed, so the overlay has of course been
moved, as is always the case.


        Stefan

Re: Info mutilates user overlays.

[Luc Teirlinck]

Stefan Monnier wrote:

   You say this is unexpected, but you don't say what you expected.
   The text of the buffer has changed, so the overlay has of course been
   moved, as is always the case.

On second thought yes.  Also, there are worse problems in Info than
this one, related to the same fact.  Position registers get treated
exactly like overlays and hence are unusable.  Bookmarks at first seem
to work, but malfunction.  After you save one bookmark, say in
(emacs)Secondary Selection, then all subsequent bookmarks in different
Info files get saved under "Secondary Selection".  The user believes
he has saved, say 10 bookmarks, but all but the last are gone and it
is saved under a misleading name.  Position registers and bookmarks are
mainly useful in large files, so the one place where they would be the
most useful (if they worked) are Info files.

In as far as Miles' question is concerned, I am using overlays to
highlight regions with a non-nil text or overlay property, but I
sometimes use overlays to highlight all kinds of stuff.  I can always
rehighlight.  That is somewhat inconvenient, but less inconvenient
than the register-bookmark stuff.

There is one more problem with that overlay repositioning.  Maybe it
applies to *Help* buffers as well.  Unreferenced deleted overlays are
inaccessible (in as far as I know) and hence are garbage, which (I
would guess) gets garbage collected.  But these repositioned overlays,
even if unreferenced, remain accessible through (overlays-in 1 2),
hence are not garbage and would (I guess) never get collected.  If
some package spews out a lot of these overlays and fails to delete
them during updates, as a result of narrowing, can that get a problem?

Sincerely,

Luc.

Re: Info mutilates user overlays.

[Luc Teirlinck]

>From my previous message:

    Bookmarks at first seem to work, but malfunction.  After you save one
    bookmark, say in (emacs)Secondary Selection, then all subsequent
    bookmarks in different Info files get saved under "Secondary
    Selection".

OK, not if you are careful and do not accept the default.

Sincerely,

Luc.

Re: Info mutilates user overlays.

[Stefan Monnier]

> this one, related to the same fact.  Position registers get treated
> exactly like overlays and hence are unusable.  Bookmarks at first seem
> to work, but malfunction.  After you save one bookmark, say in
> (emacs)Secondary Selection, then all subsequent bookmarks in different
> Info files get saved under "Secondary Selection".  The user believes

These are bugs (or at least misfeatures).  Patches welcome.

> In as far as Miles' question is concerned, I am using overlays to
> highlight regions with a non-nil text or overlay property, but I
> sometimes use overlays to highlight all kinds of stuff.  I can always
> rehighlight.  That is somewhat inconvenient, but less inconvenient
> than the register-bookmark stuff.

How do you highlight?  What do you highlight?  What for?

> There is one more problem with that overlay repositioning.  Maybe it
> applies to *Help* buffers as well.  Unreferenced deleted overlays are
> inaccessible (in as far as I know) and hence are garbage, which (I
> would guess) gets garbage collected.  But these repositioned overlays,
> even if unreferenced, remain accessible through (overlays-in 1 2),
> hence are not garbage and would (I guess) never get collected.  If
> some package spews out a lot of these overlays and fails to delete
> them during updates, as a result of narrowing, can that get a problem?

Of course it can be a problem.  Packages are expected to be careful with
such things and if necessary can use the recently mentioned `evaporate'
property to help garbage-collect them.  I think overlays should
default to the `evaporate' behavior, but it's probably too late to change
the default now.


        Stefan

Re: Info mutilates user overlays.

[Luc Teirlinck]

Stefan Monnier wrote:
   
   These are bugs (or at least misfeatures).  Patches welcome.

Actually, in the position register case, there are _two_ problems.
First of all, the position register can really get moved, just like
the overlay, but secondly, even if not, it would only be usable while
one was in the particular node, due to narrowing.  Position registers
in RMAIL do not really work very well either, in the sense that you
can not use them to jump back to a message in which you have
"registered" a position, you have to go to that message manually first
and only then can you jump to the position.  I do not really know what
the best solution here is, since in user-narrowed buffers, refusing to
go to the unreachable spot makes sense.  One definitely does not want
to use `widen', because that would look terrible in RMAIL and Info.
Bookmarks in RMAIL do not seem to work at all.  (They would probably
not be very useful as very long term things there, but what I am
saying is, you cannot use them to try to compensate for the
impossibility to use position registers.)  Bookmarks seem to work in
Info, even though one has to be careful not to accept misleading
defaults.

Sincerely,

Luc.

Re: Info mutilates user overlays.

[Luc Teirlinck]

Stefan monnier wrote:

   How do you highlight?  What do you highlight?  What for?

In the very concrete case at hand, I was highlighting regions with
non-nil `help-echo' property, using overlays with background color.
The code works for properties other than help-echo too.  Again, in the
concrete case at hand, I was doing it explicitly to check whether my
code would work in *info* buffers.  But I have used overlays
personally for a variety of reasons, for instance, to highlight for
emphasis.  I guess I could just tell in the documentation that the
highlighting can easily "evaporate" in certain buffers, such as *info*
and *help*.  The user can always re-highlight.  For my unrelated
personal uses (that can not be reconstructed from information contained
in the buffer) I might have to save my overlays in a file to do that.

Sincerely,

Luc.

Re: Info mutilates user overlays.

[Stefan Monnier]

> In the very concrete case at hand, I was highlighting regions with
> non-nil `help-echo' property, using overlays with background color.
> The code works for properties other than help-echo too.  Again, in the
> concrete case at hand, I was doing it explicitly to check whether my
> code would work in *info* buffers.  But I have used overlays
> personally for a variety of reasons, for instance, to highlight for

Please: *how* do you highlight ?
Did you create some commands similar to what is available under M-g
but using overlays, or do you add the overlay from a hook, or ...

I mean, your example using M-: (make-overlay foo bar) RET and stuff
like that, I don't really care about the interaction with info.

overlays and text-properties should basically never be added without keeping
track of them and deleting or using them later on.  Kind of like drawing
in GUIs: you have to keep track of what you've drawn to potentially update
it if the window is resized, hidden, deiconified, ...


        Stefan

Re: Info mutilates user overlays.

[Stefan Monnier]

> Actually, in the position register case, there are _two_ problems.
> First of all, the position register can really get moved, just like
> the overlay, but secondly, even if not, it would only be usable while
> one was in the particular node, due to narrowing.  Position registers

Position-registers and bookmarks should be made to work in such cases
as well.  I think a somewhat unified approach (also unified with the
help-xref-stack) would be welcome.
When I say unified, I mean at the implementation level.  Every "view"
such as an info-node, a help item, an email, ... should set
a `here-is-how-to-find-me-back' variable (that variable is called
`help-xref-stack-item' in help-mode).


        Stefan

Re: Info mutilates user overlays.

[Luc Teirlinck]

Stefan Monnier wrote:

   Please: *how* do you highlight ?
   Did you create some commands similar to what is available under M-g
   but using overlays, or do you add the overlay from a hook, or ...

The user would use some interactive command, say 
`M-x scan-buf-char-property-regions', or a key sequence bound to it.
I have a command `scan-buf-update-char-region-overlays' that updates
_all_ highlighting produced by the previous and similar commands, in
the accessible portion of the current buffer.  (Except that I still
have to make the updating work better for narrowed buffers with
different pieces highlighted differently.)  Maybe just adding the
updating command to the right hook might solve the problem.  I will
take a look whether any hook is automatically called in the given
situation.  Otherwise, the user could call the updating command, if he
wants the highlighting restored.

I keep track of the overlays by giving them a "private"
'scan-buf-char-property-regions property, so I know which overlays
were made by the function and I can delete them as necessary.

The highlighting is actually _meant_ to be temporary (to get an idea
of which text properties are present in the buffer), but it is
somewhat inconvenient if it disappears after, say, quickly following a
reference and then a quick `l' to get back.  As I said though,
updating everything is convenient for the user and could be made
automatic using some hook.

I could set a timer, but in read-only buffers that would lead to a lot
of completely needless deletion and re-creation of overlays, generating
garbage to be collected and it will normally be obvious to the user
when the buffer has changed and needs to be updated.

Sincerely,

Luc.

Re: Info mutilates user overlays.

[Luc Teirlinck]

>From my previous message:
   
   Maybe just adding the updating command to the right hook might
   solve the problem.

I will take a closer look at it, but at first sight
`Info-selection-hook' looks like it might work.  There probably are
other *info* style buffers, so I still would have to tell that in
certain situations manual re-highlighting will be necessary.

Sincerely,

Luc.

Re: Info mutilates user overlays.

[Miles Bader]

On Wed, Oct 01, 2003 at 06:18:53PM -0500, Luc Teirlinck wrote:
> I will take a closer look at it, but at first sight
> `Info-selection-hook' looks like it might work.  There probably are
> other *info* style buffers, so I still would have to tell that in
> certain situations manual re-highlighting will be necessary.

I think Stefan's comparison with window-system display updates is apropos.

Perhaps a similar solution would be in order: if there are situations where
it's desirable for overlays (&c) to be re-established after a buffer-smashing
operation, maybe there should just be a list of (FUNCTION . ARGS) entries,
and (apply FUNCTION ARGS) would be done on each entry after such an event.

Are there other places than info where a buffer gets smashed in a way that
doesn't entirely invalidate the data anyway (re-visiting a file comes to
mind)?

-Miles
-- 
Come now, if we were really planning to harm you, would we be waiting here, 
 beside the path, in the very darkest part of the forest?

Re: Info mutilates user overlays.

[Richard Stallman]

    Position-registers and bookmarks should be made to work in such cases
    as well.  I think a somewhat unified approach (also unified with the
    help-xref-stack) would be welcome.
    When I say unified, I mean at the implementation level.  Every "view"
    such as an info-node, a help item, an email, ... should set
    a `here-is-how-to-find-me-back' variable (that variable is called
    `help-xref-stack-item' in help-mode).

This extension of mechanism would be coherent, but I think it is not
the right place to direct our effort.  We could take care of this bug
much quicker by making it an error to set a bookmark in Info mode.  It
is not ideal, but not a big deal.  Other improvements in etc/TODO
would make far more difference in terms of the user's benefit.  Would
people work on some of them?