Overlay insertion types, markers, etc.

Overlay insertion types, markers, etc.

[Drew Adams]

From (elisp) `Overlays':

"An overlay uses markers to record its beginning and end."

1. You can create an overlay with `make-overlay',
passing it markers.

But the insertion types of those markers are not used
for that overlay.  Instead, you can specify those types
for the overlay as optional `make-overlay' args.  By
default, text you insert at the beginning, but not at
the end, of the overlay is included in the (extended)
overlay.

First question:

Why, if you pass markers, isn't the default to use
the insertion types of those markers?


2. `make-overlay' seems to be the only way to specify
the insertion types for an overlay.  Is that right,
or did I miss something?  `move-overlay' doesn't let
you specify them, nor does `copy-overlay'.  (And I
don't see this being handled by `overlay-put'' - they
are not overlay properties.)

There are hook properties `insert-in-front-hooks'
and `insert-behind-hooks'.  But that's not the same.

Second question:

Why isn't there (or is there?) a simple way to
change the "marker insertion types" of an existing
overlay?

Suppose you want to copy an existing overlay and
then change some things in the copy.  You can't
change the insertion types for it, right?

I guess you need to use `make-overlay', specifying
insertion types, and then explicitly copy everything
else from the first overlay.  Is that right?


3. Similarly, the default BUFFER for `make-overlay'
is the current buffer, even if you pass markers.

Third question (similar to the first):

Why, if you pass markers to `make-overlay', and
you don't pass arg BUFFER, isn't the default to
use the buffer of those markers?


4. (Repeating) "An overlay uses markers to record
its beginning and end."

It seems that an overlay "uses markers" but those
aren't the markers you passed to `make-overlay'.
Is that right?

Fourth question (also similar to the first):

Can you retrieve the markers that are "used by"
an overlay, i.e., as markers?  I don't see that
you can.  (`overlay-start' and `overlay-end',
for example, don't give you markers.)  You can
of course create suitable (but separate) markers.

(elisp) `Managing Overlays' says, about
`move-overlay':

"This is the only valid way to change the endpoints
of an overlay.  Do not try modifying the markers in
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
the overlay by hand, as that fails to update other
vital data structures and can cause some overlays
to be lost."

That makes me wonder.  I don't even see how you
could try to "modify the markers of the overlay"
(with Lisp).  How so?  How to get those markers?


I feel like I'm probably missing something here,
but I don't notice it in the manual, doc strings,
or Lisp code (only `remove-overlay' and
`copy-overlay' are coded in Lisp, it seems).

Do I understand the situation correctly?  If so,
what's the rationale for it?  I'm guessing there
are good reasons?

RE: Overlay insertion types, markers, etc.

[Drew Adams]

ping.

I really would like to know about this,
if someone has an idea.  Thx.

___

> From (elisp) `Overlays':
> 
> "An overlay uses markers to record its beginning and end."
> 
> 1. You can create an overlay with `make-overlay',
> passing it markers.
> 
> But the insertion types of those markers are not used
> for that overlay.  Instead, you can specify those types
> for the overlay as optional `make-overlay' args.  By
> default, text you insert at the beginning, but not at
> the end, of the overlay is included in the (extended)
> overlay.
> 
> First question:
> 
> Why, if you pass markers, isn't the default to use
> the insertion types of those markers?
> 
> 
> 2. `make-overlay' seems to be the only way to specify
> the insertion types for an overlay.  Is that right,
> or did I miss something?  `move-overlay' doesn't let
> you specify them, nor does `copy-overlay'.  (And I
> don't see this being handled by `overlay-put'' - they
> are not overlay properties.)
> 
> There are hook properties `insert-in-front-hooks'
> and `insert-behind-hooks'.  But that's not the same.
> 
> Second question:
> 
> Why isn't there (or is there?) a simple way to
> change the "marker insertion types" of an existing
> overlay?
> 
> Suppose you want to copy an existing overlay and
> then change some things in the copy.  You can't
> change the insertion types for it, right?
> 
> I guess you need to use `make-overlay', specifying
> insertion types, and then explicitly copy everything
> else from the first overlay.  Is that right?
> 
> 
> 3. Similarly, the default BUFFER for `make-overlay'
> is the current buffer, even if you pass markers.
> 
> Third question (similar to the first):
> 
> Why, if you pass markers to `make-overlay', and
> you don't pass arg BUFFER, isn't the default to
> use the buffer of those markers?
> 
> 
> 4. (Repeating) "An overlay uses markers to record
> its beginning and end."
> 
> It seems that an overlay "uses markers" but those
> aren't the markers you passed to `make-overlay'.
> Is that right?
> 
> Fourth question (also similar to the first):
> 
> Can you retrieve the markers that are "used by"
> an overlay, i.e., as markers?  I don't see that
> you can.  (`overlay-start' and `overlay-end',
> for example, don't give you markers.)  You can
> of course create suitable (but separate) markers.
> 
> (elisp) `Managing Overlays' says, about
> `move-overlay':
> 
> "This is the only valid way to change the endpoints
> of an overlay.  Do not try modifying the markers in
>                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> the overlay by hand, as that fails to update other
> vital data structures and can cause some overlays
> to be lost."
> 
> That makes me wonder.  I don't even see how you
> could try to "modify the markers of the overlay"
> (with Lisp).  How so?  How to get those markers?
> 
> 
> I feel like I'm probably missing something here,
> but I don't notice it in the manual, doc strings,
> or Lisp code (only `remove-overlay' and
> `copy-overlay' are coded in Lisp, it seems).
> 
> Do I understand the situation correctly?  If so,
> what's the rationale for it?  I'm guessing there
> are good reasons?

Re: Overlay insertion types, markers, etc.

[Eli Zaretskii]

> Date: Tue, 12 Nov 2019 00:09:18 +0000 (UTC)
> From: Drew Adams <drew.adams@oracle.com>
> 
> Why, if you pass markers, isn't the default to use
> the insertion types of those markers?

Because the markers passed to make-overlay are treated only as
representing buffer positions, similarly to other functions, like
goto-char.

> 2. `make-overlay' seems to be the only way to specify
> the insertion types for an overlay.  Is that right,
> or did I miss something?

That's right.

> Why isn't there (or is there?) a simple way to
> change the "marker insertion types" of an existing
> overlay?

Because it's easy to do that by hand?

> Suppose you want to copy an existing overlay and
> then change some things in the copy.  You can't
> change the insertion types for it, right?
> 
> I guess you need to use `make-overlay', specifying
> insertion types, and then explicitly copy everything
> else from the first overlay.  Is that right?

Yes.

> 3. Similarly, the default BUFFER for `make-overlay'
> is the current buffer, even if you pass markers.
> 
> Third question (similar to the first):
> 
> Why, if you pass markers to `make-overlay', and
> you don't pass arg BUFFER, isn't the default to
> use the buffer of those markers?

Because that just complicates the implementation with no real gain:
what to do if the markers point to different buffers, or if their
buffers don't exist, or the position information doesn't fit, etc.?

> 4. (Repeating) "An overlay uses markers to record
> its beginning and end."
> 
> It seems that an overlay "uses markers" but those
> aren't the markers you passed to `make-overlay'.
> Is that right?

Yes.

> Can you retrieve the markers that are "used by"
> an overlay, i.e., as markers?

No.  We don't want to give Lisp access to those markers, as that could
mean giving to long a rope to Lisp programs to hang themselves.

> "This is the only valid way to change the endpoints
> of an overlay.  Do not try modifying the markers in
>                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> the overlay by hand, as that fails to update other
> vital data structures and can cause some overlays
> to be lost."
> 
> That makes me wonder.  I don't even see how you
> could try to "modify the markers of the overlay"
> (with Lisp).

I believe that remark is for C programming.

RE: Overlay insertion types, markers, etc.

[Drew Adams]

Thanks very much for answering.  I have a few
follow-up questions, for clarity.  I don't expect
to belabor this, but things are still not very
clear to me.

> > Why, if you pass markers, isn't the default to use
> > the insertion types of those markers?
> 
> Because the markers passed to make-overlay are treated
> only as representing buffer positions, similarly to
> other functions, like goto-char.

But _why_ are they used only for their positions?
That's the question.

Functions like `goto-char' have no use of the other
information (buffer, insertion types) in a marker,
besides its position.  An overlay does have an
associated buffer and insertion types.

(I suppose someone might suggest that `goto-char'
could go to a marker in another buffer, and that
it could accept an optional BUFFER arg.  But I
haven't done that.  In any case, surely it could
have no use for an insertion type, right?)

Why doesn't `make-overlay' get the defaults for
such things from its input markers?

> > 2. `make-overlay' seems to be the only way to specify
> > the insertion types for an overlay.  Is that right,
> > or did I miss something?
> 
> That's right.
> 
> > Why isn't there (or is there?) a simple way to
> > change the "marker insertion types" of an existing
> > overlay?
> 
> Because it's easy to do that by hand?

How so?  How do you change the insertion types of
an _existing_ overlay (by hand or otherwise)?
I don't see that possibility.  And below you seem
to point to using C (?).  I'm asking whether (and
if so how) you can do this with Lisp.

> > Suppose you want to copy an existing overlay and
> > then change some things in the copy.  You can't
> > change the insertion types for it, right?
> >
> > I guess you need to use `make-overlay', specifying
> > insertion types, and then explicitly copy everything
> > else from the first overlay.  Is that right?
> 
> Yes.

Is there a good reason not to provide a way to
change insertion types in an existing overlay?

(Assuming there is no such way - see above: not
clear to me what doing so "by hand" means, i.e.,
how to do it.)

You can change overlay properties of an existing
overlay. And you can change its limits
(`move-overlay').  But you can't change its
insertion types or its buffer (or can you?).
Why not provide a way to do that?

Wouldn't it be useful to even have setter
functions for an overlay, which correspond to the
getter functions (`overlay-start', `overlay-end',
`overlay-buffer')?  Or even be able to use `setf'
with such getters?

I'm not so much proposing a change.  To start
with, I really want to understand the design and
its logic.

> > 3. Similarly, the default BUFFER for `make-overlay'
> > is the current buffer, even if you pass markers.
> >
> > Third question (similar to the first):
> >
> > Why, if you pass markers to `make-overlay', and
> > you don't pass arg BUFFER, isn't the default to
> > use the buffer of those markers?
> 
> Because that just complicates the implementation with no real gain:
> what to do if the markers point to different buffers, or if their
> buffers don't exist, or the position information doesn't fit, etc.?

(This is about determining the _default_ buffer
value, when the BUFFER arg is not provided.)

There are various possibilities (design choices).
Off the top of my head, naively:

One obvious possibility is to just do what we do
now, if the markers don't provide usable buffer
info: use the current buffer.

If the markers point to different buffers, for
example, just ignore those buffers - just do what
we do now (use current buffer).

If one limit passed to `make-overlay' is a marker
and the other is a number, use the buffer of the
marker.

If a marker has no buffer then it also has no
position, no?  How could it not?  So I don't get
your question "if their buffers don't exist".

If a marker has a position then that position must
"fit" its buffer, no?  How could it not?  So I don't
get your question "if the position information
doesn't fit".

But again, if for some reason the info from a marker
isn't usable, then just don't use it - do what we
do now.

> > 4. (Repeating) "An overlay uses markers to record
> > its beginning and end."
> >
> > It seems that an overlay "uses markers" but those
> > aren't the markers you passed to `make-overlay'.
> > Is that right?
> 
> Yes.
> 
> > Can you retrieve the markers that are "used by"
> > an overlay, i.e., as markers?
> 
> No.  We don't want to give Lisp access to those markers,
> as that could mean giving to long a rope to Lisp programs
> to hang themselves.

I don't see what the problem is.  They're just
markers (presumably).  Any code can use any marker
to hang to hang itself now.  How is that related to
the fact that an overlay would use such a marker?

> > "This is the only valid way to change the endpoints
> > of an overlay.  Do not try modifying the markers in
> >                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > the overlay by hand, as that fails to update other
> > vital data structures and can cause some overlays
> > to be lost."
> >
> > That makes me wonder.  I don't even see how you
> > could try to "modify the markers of the overlay"
> > (with Lisp).
> 
> I believe that remark is for C programming.

So in the Elisp manual we tell users not to try
to use C to modify the markers in an overlay
"by hand"?

Why is such a remark appropriate/helpful?  How
would a reader even guess that C programming is
what that remark has in mind?
___

Please note: I have nothing particular in mind
here - no agenda.  I'm trying to understand what
the case is, and why.

Depending on the "why" (once I understand it),
if it seems weak then I may also wonder "why not"
do something more helpful/useful.

So far, that's what I'm wondering about things
like obtaining the insertion types and buffer
from the markers used to define an overlay.

But I have no special use case in mind.  It just
seems (so far) like something useful is missing.

I'm guessing that I'm wrong about this, just
because I don't yet understand enough why things
are the way they are.  Is this maybe just a case
of no one ever getting around to improve it, or
are there good reasons why things are the way
they are?

Re: Overlay insertion types, markers, etc.

[Eli Zaretskii]

> Date: Fri, 15 Nov 2019 08:54:05 -0800 (PST)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: emacs-devel@gnu.org
> 
> > Because the markers passed to make-overlay are treated
> > only as representing buffer positions, similarly to
> > other functions, like goto-char.
> 
> But _why_ are they used only for their positions?

Because that's the minimal useful thing to do.  We do that in many
other APIs.  Anything beyond that would need a good reason.

> > > Why isn't there (or is there?) a simple way to
> > > change the "marker insertion types" of an existing
> > > overlay?
> > 
> > Because it's easy to do that by hand?
> 
> How so?  How do you change the insertion types of
> an _existing_ overlay (by hand or otherwise)?

You make a new overlay and copy whatever you want to keep from the old
one.

> Is there a good reason not to provide a way to
> change insertion types in an existing overlay?

I don't know.  Probably because a need never aroused.

> You can change overlay properties of an existing
> overlay. And you can change its limits
> (`move-overlay').  But you can't change its
> insertion types or its buffer (or can you?).
> Why not provide a way to do that?

We don't normally provide every possible way to do every possible
change.  Just those which are needed.

> I'm not so much proposing a change.  To start
> with, I really want to understand the design and
> its logic.

The design was b y Richard, so he is the only one who might remember
why he didn't go beyond what we have.

> > > Why, if you pass markers to `make-overlay', and
> > > you don't pass arg BUFFER, isn't the default to
> > > use the buffer of those markers?
> > 
> > Because that just complicates the implementation with no real gain:
> > what to do if the markers point to different buffers, or if their
> > buffers don't exist, or the position information doesn't fit, etc.?
> 
> (This is about determining the _default_ buffer
> value, when the BUFFER arg is not provided.)
> 
> There are various possibilities (design choices).
> Off the top of my head, naively:

Sure, there is.  But they all complicate things, whereas passing a
buffer is simple and efficient.

> > > Can you retrieve the markers that are "used by"
> > > an overlay, i.e., as markers?
> > 
> > No.  We don't want to give Lisp access to those markers,
> > as that could mean giving to long a rope to Lisp programs
> > to hang themselves.
> 
> I don't see what the problem is.  They're just
> markers (presumably).  Any code can use any marker
> to hang to hang itself now.  How is that related to
> the fact that an overlay would use such a marker?

Giving Lisp programs access to those markers will complicate related
features.  For example, the display engine currently tracks only
changes to overlays, it doesn't track changes to its markers.  Also,
having references to those markers lying around will complicate how
overlays are deleted -- currently we just delete their markers.  And
these are just results of 5 sec of thinking.

> > I believe that remark is for C programming.
> 
> So in the Elisp manual we tell users not to try
> to use C to modify the markers in an overlay
> "by hand"?
> 
> Why is such a remark appropriate/helpful?  How
> would a reader even guess that C programming is
> what that remark has in mind?

I don't know.  I certainly find it helpful.

> But I have no special use case in mind.  It just
> seems (so far) like something useful is missing.

I don't think anything potentially useful is missing.  You can do
everything you want with this stuff, albeit sometimes indirectly.

> I'm guessing that I'm wrong about this, just
> because I don't yet understand enough why things
> are the way they are.  Is this maybe just a case
> of no one ever getting around to improve it, or
> are there good reasons why things are the way
> they are?

I don't know.

RE: Overlay insertion types, markers, etc.

[Drew Adams]

> The design was by Richard, so he is the only one who might remember
> why he didn't go beyond what we have.

Ccing Richard, in case he might want to add something.

Re: Overlay insertion types, markers, etc.

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > The design was by Richard, so he is the only one who might remember
  > > why he didn't go beyond what we have.

  > Ccing Richard, in case he might want to add something.

I have not been following this thread.  What are the changes people
propose?

It is possible that the answer is simple -- that I didn't see a need
to do the work to add any other features in that area.  But unless
I know what is proposed, I can't know the answer.

-- 
Dr Richard Stallman
Founder, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

RE: Overlay insertion types, markers, etc.

[Drew Adams]

>   > > The design was by Richard, so he is the only one who might
> remember
>   > > why he didn't go beyond what we have.
> 
>   > Ccing Richard, in case he might want to add something.
> 
> I have not been following this thread.  What are the changes people
> propose?
> 
> It is possible that the answer is simple -- that I didn't see a need
> to do the work to add any other features in that area.  But unless
> I know what is proposed, I can't know the answer.

Thanks for replying.

No changes to anything were proposed.  I asked some
questions, trying to understand the design.  This
is all in the first message of the thread (and the
thread is short).  This the URL of that message:

https://lists.gnu.org/archive/html/emacs-devel/2019-11/msg00414.html

Eli responded to some of the questions.  I replied
with clarification questions, to understand his
reply.  But essentially everything I'm asking about
is in the initial message.

In summary, these are the questions:

1. Why, if you pass markers [to `make-overlay],
   isn't the default to use the insertion types of
   those markers?

2. Why isn't there (or is there?) a simple way to
   change the "marker insertion types" of an
   existing overlay?

3. Why, if you pass markers to `make-overlay', and
   you don't pass arg BUFFER, isn't the default to
   use the buffer of those markers?

4. Can you retrieve the markers that are "used by"
   an overlay, i.e., as markers?

Re: Overlay insertion types, markers, etc.

[Stefan Monnier]

> 4. Can you retrieve the markers that are "used by"
>    an overlay, i.e., as markers?

No, you can't, and you shouldn't be able to: it's an implementation
detail, that may change in the future (IIUC the `noverlays` branch
indeed makes this change as part of a rewrite which tries to eliminate
some algorithmic complexity problems in the overlay code).

Also it would introduce all kinds of extra work:
- when an overlay is moved, the redisplay code is told about it, whereas
  no redisplay is normally caused by moving a marker, so we'd need to
  figure out what to do when an "overlay-owned" marker is moved.
- the user could now move the start-marker of an overlay to another buffer
  than the end-marker
- the user could now move the end-marker of an overlay before the
  start-marker (this can already happen but only in very specific
  cases, so we'd have to handle it in more cases).
- ...


        Stefan

RE: Overlay insertion types, markers, etc.

[Drew Adams]

> > 4. Can you retrieve the markers that are "used by"
> >    an overlay, i.e., as markers?
> 
> No, you can't, and you shouldn't be able to: it's
> an implementation detail, that may change in the
> future (IIUC the `noverlays` branch indeed makes
> this change as part of a rewrite which tries to
> eliminate some algorithmic complexity problems in
> the overlay code).

OK.  The other questions I asked were about using
the markers passed to `make-overlay' to specify the
_defaults_ for the overlay: insertion types and
buffer.  That is, the values to use for those if
the optional args to specify them aren't provided.

Those other questions were not about the markers in
the overlay itself - they were only about obtaining
default info when creating an overlay and passing
it markers instead of numbers.

This question (#4) was different.  It was about
Lisp access to the markers used in the overlay
itself.  About which nothing is said, BTW.  But Eli
has confirmed that they are indeed markers.

"Marker" in the Elisp doc (other than here) means
an object accessible from Lisp.  It's confusing
to talk, in passing - with no description or
explanation, of "markers" that are used by an
overlay (i.e., as part of it), but that are
(it turns out) not markers in the usual sense,
because they're not accessible to Lisp.

It confused me anyway.  All the more so because
the doc doesn't actually say they're only internal,
not accessible.

Hence this question (#4), to find out if they are
in fact somehow accessible.

> Also it would introduce all kinds of extra work:
> - when an overlay is moved, the redisplay code is
>   told about it, whereas no redisplay is normally
>   caused by moving a marker, so we'd need to figure
>   out what to do when an "overlay-owned" marker is
>   moved.

I'm not proposing anything.  But I'm guessing that
that point is incorrect in one sense (one direction).

If the overlay does, as the doc and Eli say, use
markers, then when the overlay is moved those
markers are presumably already updated accordingly.

But there's also the reverse, which you bring up:
moving the markers.  You segue to moving (e.g. by
Lisp) one or both of the overlay markers.

> - the user could now move the start-marker of an
>   overlay to another buffer than the end-marker
> - the user could now move the end-marker of an
>   overlay before the start-marker (this can already
>   happen but only in very specific cases, so we'd
>   have to handle it in more cases).

Those are cases of not only examining the markers
from Lisp but changing them.

There is already a (mysterious, IMO, since the
markers are currently internal) warning in the
Elisp manual _not to do_ that:

  "Do not try modifying the markers in the
   overlay by hand..."

(Eli has said that that warning must be to not
try that using C code.  Why such a warning would
be in the Elisp manual is not clear to me.)

But suppose the markers were modifiable from Lisp,
and suppose someone ignored the warning.  The
result could (by design) simply be to delete the
overlay.  IOW, if an overlay for any reason has
markers that are nonsensical (for an overlay) then
treat it as deleted (no buffer).  One possibility.

Again, I didn't propose providing Lisp access.
I was asking why we tell users that an overlay
has markers, instead of, for example, just saying
that it has a start, an end, and a buffer?  Why
talk about the "markers" of an overlay at all?

Especially since they apparently are not markers
in the usual sense (Lisp objects).

We don't use markers passed as `make-overlay'
args for anything but initializing the start and
end positions (presumably swapping them, if
passed in the wrong order numerically).

What's the point of documenting that an overlay
has markers?  And not saying clearly that they're
only internal - not normal markers (i.e.,
Lisp-accessible)?  And warning users not to
modify them - which they can't do anyway using
Lisp.

It's the doc that led me to ask these questions.

I haven't proposed any changes to the behavior -
or even to the doc.  But given the answers to my
questions so far, it seems to me (assuming that
no behavior changes are in order or desired),
that the doc should maybe be changed a bit.  No?

---

BTW, you mention "the `noverlays` branch indeed
makes this change as part of a rewrite which tries
to eliminate some algorithmic complexity problems
in the overlay code."

What's that about?  Is there a plan to remove
overlays?  It would be good to know this, so we
(e.g. 3rd-party code) don't waste time developing
things that depend more on overlays.

Re: Overlay insertion types, markers, etc.

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > 1. Why, if you pass markers [to `make-overlay],
  >    isn't the default to use the insertion types of
  >    those markers?

I don't think I ever thought about the question back then.

Did pointers even have insertion types, back when overlays were first
introduced?

However, the function calling convention would have to bec omplex
to give you three options: yes, no, or inherit from the marker.

  > 2. Why isn't there (or is there?) a simple way to
  >    change the "marker insertion types" of an
  >    existing overlay?

I never thought of adding it.

  > 3. Why, if you pass markers to `make-overlay', and
  >    you don't pass arg BUFFER, isn't the default to
  >    use the buffer of those markers?

I never thought about it.

  > 4. Can you retrieve the markers that are "used by"
  >    an overlay, i.e., as markers?

If you could get at them, you could change their buffer positions.
I think that would mess up the overlay sort order and cause bugs.
You could also put them in another buffer  and cause even worse trouble.

-- 
Dr Richard Stallman
Founder, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Overlay insertion types, markers, etc.

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: eliz@gnu.org, emacs-devel@gnu.org
> Date: Wed, 20 Nov 2019 11:17:04 -0500
> 
>   > 1. Why, if you pass markers [to `make-overlay],
>   >    isn't the default to use the insertion types of
>   >    those markers?
> 
> I don't think I ever thought about the question back then.
> 
> Did pointers even have insertion types, back when overlays were first
> introduced?

No.  insertion-type of markers was added in Emacs 19.30, and overlays
were added before that.

RE: Overlay insertion types, markers, etc.

[Drew Adams]

>   > 1. Why, if you pass markers [to `make-overlay],
>   >    isn't the default to use the insertion types of
>   >    those markers?
> 
> I don't think I ever thought about the question back then.
> 
> Did pointers even have insertion types, back when
> overlays were first introduced?

No idea.  I know much less than you about this.

> However, the function calling convention would have
> to be complex to give you three options: yes, no,
> or inherit from the marker.

Not sure what you mean.  We already have the
ability to specify, using optional args, the
insertion types and the buffer.

The only question is where the _defaults_ for
those come from, i.e., when you don't specify
their optional args.

Today, the defaults ignore such info from any
marker args.  Today, the current buffer is the
default buffer, and the default for insertion
is "the overlay extends to include any text
inserted at the beginning, but not text inserted
at the end".

If we instead allowed the defaults to be taken
from marker args, and if someone wanted today's
default behavior, s?he would just pass a marker
position (`marker-position') instead of the
marker itself.

And as I mentioned to Eli, if two markers are
passed and they disagree wrt such things, then
the defaulting could be done as it is now:
ignore conflicting info from the markers.

E.g., if two markers have the same buffer (or
if only one of the position args passed is a
marker), use that `marker-buffer'.  Similarly,
for FRONT-ADVANCE and REAR-ADVANCE.

And yes, such a change in defaulting would
not be backward compatible - code that passes
markers and expects the default buffer and
default insertion behavior to be as now would
break.  Users would need to update such code
to use `marker-position'.

But again, I wasn't proposing that or anything
else.  I was just trying to understand why we
do what we do.

>   > 2. Why isn't there (or is there?) a simple way to
>   >    change the "marker insertion types" of an
>   >    existing overlay?
> 
> I never thought of adding it.
> 
>   > 3. Why, if you pass markers to `make-overlay', and
>   >    you don't pass arg BUFFER, isn't the default to
>   >    use the buffer of those markers?
> 
> I never thought about it.
> 
>   > 4. Can you retrieve the markers that are "used by"
>   >    an overlay, i.e., as markers?
> 
> If you could get at them, you could change their buffer positions.
> I think that would mess up the overlay sort order and cause bugs.
> You could also put them in another buffer and cause even worse
> trouble.

That's what Stefan said.  See my reply to him
about that:

https://lists.gnu.org/archive/html/emacs-devel/2019-11/msg00603.html

In particular, I think it's confusing for the
doc to talk about the _markers of an overlay_
(this is not about the markers that you pass to
`make-overlay'), since they are not markers in
the usual sense of being Lisp objects (that is,
visible and accessible from Lisp).

At least it confused me and prompted my questions.

Now that I understand a bit more, I'd suggest
that the doc be changed to remove mention of an
overlay having markers (internal though they are).
That doesn't add anything, I think, and it can
confuse.

It should be enough to say that, like a marker,
an overlay has a buffer and text-insertion
behavior.  In particular, the latter means that,
by default, an overlay advances when you insert
text before it.