bug#20253: 24.4; Document `region' face behavior changes, overlay, priority

bug#20253: 24.4; Document `region' face behavior changes, overlay, priority

[Drew Adams]

Since Emacs 24.3, if I recall correctly, face `region' uses an overlay.
This produces changes in the user-observable behavior - that's why the
change was made.

But the NEWS for 24.3 and for 24.4 do not mention anything about this,
AFAICT.  Not only does NEWS not seem to say anything about `region'
using an overlay, but it does not even mention the observable behavior
differences that this produces.

And the Elisp manual is likewise silent on this change.

Please tell users that such a change was made.  That's what NEWS is for.

And please tell users, in the Elisp manual, what the default priority of
the overlay is.  Testing shows a priority value of (nil . 100), which is
not even a documented form that an overlay priority can take.  But even
if that is what the value needs to be, for internal reasons, at least
tell users what it is.

And if, for some reason, users are supposed not to change the priority,
then say so, in the doc.  And if there is no such restriction or
recommendation, then let them know, if they should be assigning a
priority value that is a cons, such as (nil . NUMBER).

IOW, please communicate with your users about such changes: what they
mean for users (what behavior differences they can expect), and how they
can manipulate/customize the behavior.

In GNU Emacs 24.4.1 (i686-pc-mingw32)
 of 2014-10-24 on LEG570
Windowing system distributor `Microsoft Corp.', version 6.1.7601
Configured using:
 `configure --prefix=/c/usr'

bug#20253: 24.4; Document `region' face behavior changes, overlay, priority

[Alex]

Drew Adams <drew.adams@oracle.com> writes:

> Since Emacs 24.3, if I recall correctly, face `region' uses an overlay.
> This produces changes in the user-observable behavior - that's why the
> change was made.
>
> But the NEWS for 24.3 and for 24.4 do not mention anything about this,
> AFAICT.  Not only does NEWS not seem to say anything about `region'
> using an overlay, but it does not even mention the observable behavior
> differences that this produces.
>
> And the Elisp manual is likewise silent on this change.
>
> Please tell users that such a change was made.  That's what NEWS is for.
>
> And please tell users, in the Elisp manual, what the default priority of
> the overlay is.  Testing shows a priority value of (nil . 100), which is
> not even a documented form that an overlay priority can take.  But even
> if that is what the value needs to be, for internal reasons, at least
> tell users what it is.
>
> And if, for some reason, users are supposed not to change the priority,
> then say so, in the doc.  And if there is no such restriction or
> recommendation, then let them know, if they should be assigning a
> priority value that is a cons, such as (nil . NUMBER).
>
> IOW, please communicate with your users about such changes: what they
> mean for users (what behavior differences they can expect), and how they
> can manipulate/customize the behavior.

The Elisp manual does mention that "Emacs sometimes uses non-numeric
priority values for some of its internal overlays", but it would be nice
if it documented what those were and what they mean.

The comments in simple.el mention that a cons cell value represents
primary/secondary priorities, but it's not clear what the difference is.

bug#20253: 24.4; Document `region' face behavior changes, overlay, priority

[Eli Zaretskii]

> From: Alex <agrambot@gmail.com>
> Date: Sun, 11 Sep 2016 17:24:45 -0600
> Cc: 20253@debbugs.gnu.org
> 
> The Elisp manual does mention that "Emacs sometimes uses non-numeric
> priority values for some of its internal overlays", but it would be nice
> if it documented what those were and what they mean.
> 
> The comments in simple.el mention that a cons cell value represents
> primary/secondary priorities, but it's not clear what the difference is.

See compare_overlays in buffer.c.

The ELisp manual is not supposed to be a comprehensive documentation
of the Emacs internals, it's supposed to provide information for users
who write Lisp programs.  People who develop Emacs itself will need to
read the sources anyway.  So it's not a big deal if some obscure
feature is not described in the manual, it just means it is not
supposed to be used by Lisp programs that are not part of Emacs.

bug#20253: 24.4; Document `region' face behavior changes, overlay, priority

[Drew Adams]

> > The Elisp manual does mention that "Emacs sometimes uses non-numeric
> > priority values for some of its internal overlays", but it would be nice
> > if it documented what those were and what they mean.
> >
> > The comments in simple.el mention that a cons cell value represents
> > primary/secondary priorities, but it's not clear what the difference is.
> 
> See compare_overlays in buffer.c.
> 
> The ELisp manual is not supposed to be a comprehensive documentation
> of the Emacs internals, it's supposed to provide information for users
> who write Lisp programs.  People who develop Emacs itself will need to
> read the sources anyway.  So it's not a big deal if some obscure
> feature is not described in the manual, it just means it is not
> supposed to be used by Lisp programs that are not part of Emacs.

I disagree that THIS is an "internal" and "obscure" feature, and
that "it is not supposed to be used by Lisp programs that are not
part of Emacs".  It cannot be, if users want to control the behavior
of the `region' overlay and similar.

Questions about this arise from end users.

The change in behavior to not have highlighting of the selected text
(region) take precedence over other highlighting has provoked some
users to ask how they can get the previous behavior again, etc.
More generally, it is a fairly common question how to juggle overlay
priorities to get the effect of what (nil . NUMBER) provides.

Such questions, and their answers, immediately lead users to this
feature (after someone digs it out for them).  They ask about
understanding and controlling the _behavior_.  They end up needing
to know about (nil . NUMBER) - but it is not something that they
started out wondering about.

This is not about some curious user wanting to delve into "Emacs
internals".

And the question of this bug report is not only about documenting
(nil . NUMBER).  It is about documenting the user-visible _behavior_
change for region highlighting, and how to accommodate and control it.

 >> IOW, please communicate with your users about such changes: what
 >> they mean for users (what behavior differences they can expect),
 >> and how they can manipulate/customize the behavior.

It's the least that users can expect from the extensible, customizable,
self-documenting editor.

bug#20253: 24.4; Document `region' face behavior changes, overlay, priority

[Eli Zaretskii]

> Date: Mon, 12 Sep 2016 10:19:42 -0700 (PDT)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: drew.adams@oracle.com, 20253@debbugs.gnu.org
> 
> I disagree that THIS is an "internal" and "obscure" feature, and
> that "it is not supposed to be used by Lisp programs that are not
> part of Emacs".  It cannot be, if users want to control the behavior
> of the `region' overlay and similar.

Of course, you disagree: you filed the bug, didn't you?

> Questions about this arise from end users.

I didn't see any.  Please invite those users to ask the questions here
or on emacs-devel.  Maybe I will change my mind after hearing them.

> And the question of this bug report is not only about documenting
> (nil . NUMBER).  It is about documenting the user-visible _behavior_
> change for region highlighting, and how to accommodate and control it.

I said nothing about that part.

bug#20253: 24.4; Document `region' face behavior changes, overlay, priority

[Alex]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Mon, 12 Sep 2016 10:19:42 -0700 (PDT)
>> From: Drew Adams <drew.adams@oracle.com>
>> Cc: drew.adams@oracle.com, 20253@debbugs.gnu.org
>>
>> Questions about this arise from end users.
>
> I didn't see any.  Please invite those users to ask the questions here
> or on emacs-devel.  Maybe I will change my mind after hearing them.
>

My curiosity about the meaning of a cons cell priority is what pushed me
to find and respond to this bug, so I guess that would count as one
question.

Since the region overlay is a very common overlay, I think it is
important enough to at least have a passing statement about its priority
value type in the documentation. Even something like "For example, some
overlays use a cons cell priority (PRIMARY . SECONDARY), where SECONDARY
is used as a tie-breaker if the PRIMARY priorities and boundaries of the
overlays are equal." would be nice.

bug#20253: 24.4; Document `region' face behavior changes, overlay, priority

[Eli Zaretskii]

> From: Alex <agrambot@gmail.com>
> Cc: Drew Adams <drew.adams@oracle.com>,  20253@debbugs.gnu.org
> Date: Wed, 21 Sep 2016 12:11:08 -0600
> 
> My curiosity about the meaning of a cons cell priority is what pushed me
> to find and respond to this bug, so I guess that would count as one
> question.
> 
> Since the region overlay is a very common overlay, I think it is
> important enough to at least have a passing statement about its priority
> value type in the documentation. Even something like "For example, some
> overlays use a cons cell priority (PRIMARY . SECONDARY), where SECONDARY
> is used as a tie-breaker if the PRIMARY priorities and boundaries of the
> overlays are equal." would be nice.

I hoped to hear some practical reasons for this, not just curiosity.
Like practical use cases where knowing that internal detail (as
opposed to using the documented methods of comparing priorities etc.)
is imperative for that use case.

People often ask questions here out of curiosity about the display
engine's inner workings, for example, and I try to answer them as best
as I can.  But that doesn't mean all I write here in those discussions
should be in the manual.

IOW, the need to have some internal detail described in the manual
(which implies we will have to update and maintain it for the years to
come) should have more important reasons than just curiosity.

bug#20253: 24.4; Document `region' face behavior changes, overlay, priority

[Alex]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Alex <agrambot@gmail.com>
>> Cc: Drew Adams <drew.adams@oracle.com>,  20253@debbugs.gnu.org
>> Date: Wed, 21 Sep 2016 12:11:08 -0600
>> 
>> My curiosity about the meaning of a cons cell priority is what pushed me
>> to find and respond to this bug, so I guess that would count as one
>> question.
>> 
>> Since the region overlay is a very common overlay, I think it is
>> important enough to at least have a passing statement about its priority
>> value type in the documentation. Even something like "For example, some
>> overlays use a cons cell priority (PRIMARY . SECONDARY), where SECONDARY
>> is used as a tie-breaker if the PRIMARY priorities and boundaries of the
>> overlays are equal." would be nice.
>
> I hoped to hear some practical reasons for this, not just curiosity.
> Like practical use cases where knowing that internal detail (as
> opposed to using the documented methods of comparing priorities etc.)
> is imperative for that use case.

I should have mentioned that aside from curiosity I did need to know
about what the region overlay's priority represented so that I can know
what priority to give overlays that interacted with the region overlay.

> People often ask questions here out of curiosity about the display
> engine's inner workings, for example, and I try to answer them as best
> as I can.  But that doesn't mean all I write here in those discussions
> should be in the manual.

Right, but I think that the region overlay is prominent enough to have
all of its properties be in a documented form.

> IOW, the need to have some internal detail described in the manual
> (which implies we will have to update and maintain it for the years to
> come) should have more important reasons than just curiosity.

Hopefully the above is enough to warrant an extra sentence or two in the
manual.

bug#20253: 24.4; Document `region' face behavior changes, overlay, priority

[Drew Adams]

Related: http://emacs.stackexchange.com/q/17324/105

(This has been reported and discussed before -
both negative values and dotted-pair values.)

bug#20253: 24.4; Document `region' face behavior changes, overlay, priority

[Eli Zaretskii]

> From: Alex <agrambot@gmail.com>
> Cc: drew.adams@oracle.com,  20253@debbugs.gnu.org
> Date: Wed, 21 Sep 2016 13:10:43 -0600
> 
> Hopefully the above is enough to warrant an extra sentence or two in the
> manual.

Done.

bug#20253: 24.4; Document `region' face behavior changes, overlay, priority

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> Hopefully the above is enough to warrant an extra sentence or two in the
>> manual.
>
> Done.

I don't think there's anything more to do here, so I'm closing this bug
report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no