bug#6935: 24.0.50; doc for `font-lock-maximum-decoration'

bug#6935: 24.0.50; doc for `font-lock-maximum-decoration'

[Drew Adams]

The doc for `font-lock-maximum-fontification' is not very
clear/complete.
 
1. For one thing, the Emacs manual deals with it only using a `setq'
example:
 
  (setq font-lock-maximum-decoration
        '((c-mode . 1) (c++-mode . 1)))
 
We should tell users how they can use Customize for customizing it.
(No, it is not obvious how to do that.)  We should not be privileging
Lisp code in .emacs this way - especially fairly complex Lisp code.
 

2. The doc string and the Customize help for it (same thing) do not help
much either.  In particular, they are missing the info that if you add
an entry for one or more modes, then you will likely want to also add
a catch-all entry for all other modes.
 
The latter notion is not presented explicitly, but the example given
does help in this regard.  One problem is that the choice of `all' as
the UI label gives the impression that it might override other,
mode-specific entries.  For example, you might well think that order is
important and that an entry of `all' overrides any other entries that
follow.  `all' should really be renamed something that conveys the fact
that it means `all other modes', not `all modes'.
 

3. Also, to understand what the choice of `level' means here, users need
to know about fontification levels.  At least a minimum of info about
that needs to be presented in the Customize (= doc string) help.
 
 
In GNU Emacs 24.0.50.1 (i386-mingw-nt5.1.2600)
 of 2010-08-16 on 3249CTO
Windowing system distributor `Microsoft Corp.', version 5.1.2600
configured using `configure --with-gcc (4.4) --no-opt --cflags
-Ic:/imagesupport/include'
 

bug#6935: 24.0.50; doc for `font-lock-maximum-decoration'

[Stefan Monnier]

> The doc for `font-lock-maximum-fontification' is not very
> clear/complete.
 
FWIW, I think this customization option (and the whole notion of level)
should disappear,


        Stefan

bug#6935: 24.0.50; doc for `font-lock-maximum-decoration'

[Drew Adams]

> > The doc for `font-lock-maximum-fontification' is not very
> > clear/complete.
>  
> FWIW, I think this customization option (and the whole notion 
> of level) should disappear,

Why?  And why just say _you think so_, with no supporting argument?  I might
think that all alligators should be painted yellow with blue polka-dots, but why
should anyone care if I give no reason?

This customization option lets users easily pick the level they want.  Just
yesterday I had a user report about this wrt Dired+ (my code had a bug that was
preventing such a choice).  Users have different needs and preferences, and they
do care about them.

Some means to control fontification level should be offered to users, and an
option is a good way to do that.  Doing away with "the whole notion of level"
would mean giving users no choice.  One size does _not_ fit all.  It's about the
users.

In any case, this bug report is about specific, doc problems wrt this feature.
Let's please address those.  The feature works, and it always has.  The doc is
not perfect; that's all.

bug#6935: 24.0.50; doc for `font-lock-maximum-decoration'

[Stefan Monnier]

> This customization option lets users easily pick the level they want.
> Just yesterday I had a user report about this wrt Dired+ (my code had
> a bug that was preventing such a choice).  Users have different needs
> and preferences, and they do care about them.

2 reasons:
- rather than let people work around problems by reducing the level, we
  should work harder to make sure that the default level is OK
  for everyone.
- "level" is the wrong way to think about this.  Instead, we should have
  separate controls for different font-lock features.


        Stefan

bug#6935: 24.0.50; doc for `font-lock-maximum-decoration'

[Drew Adams]

> > This customization option lets users easily pick the level 
> > they want.  Just yesterday I had a user report about this
> > wrt Dired+ (my code had a bug that was preventing such a
> > choice).  Users have different needs
> > and preferences, and they do care about them.
> 
> 2 reasons:
> - rather than let people work around problems by reducing the 
>   level,

It's not a question of "working around" anything.  It's about providing levels
to give users a choice.

>   we should work harder to make sure that the default level is OK
>   for everyone.

That's silly.  If there is a _default_ level then there are level choices and a
notion of level.

No one disagrees that the default level should provide minimal fontification.
The point is to allow users to move to a higher level if they so wish.

> - "level" is the wrong way to think about this.  Instead, we 
>   should have separate controls for different font-lock features.

Be specific.  If you are agreeing that users should have choice and control when
it comes to the degree of font-locking, and you just disagree with the current
"level" mechanism, then propose something specific.

Note that in the case I cited the user had the ability to fine-tune
fontification, for example by customizing individual faces.  But he wanted a
coarse-grain control, to change the _level_.

That was so even though he was unaware of the current level mechanism.  He
sought a way to affect the level of fontification in one fell swoop.  And that's
just what the current level mechanism offers.

If you have a better way to give users that possibility, then please propose it
specifically (on emacs-devel).

I'd certainly agree that the current mechanism does not give users a way to
define the makeup of a given level themselves, and that that is a limitation.
The makeup of each level is hard-coded.  I, for one, would welcome a feature
that lets users (easily) define what each level should be.

That would likely mean letting them customize the font-lock keywords for each
level in some way.  And currently there is no good way for users to use
Customize to tailor a set of font-lock keywords.

bug#6935: 24.0.50; doc for `font-lock-maximum-decoration'

[Stefan Monnier]

>> we should work harder to make sure that the default level is OK
>> for everyone.

> That's silly.  If there is a _default_ level then there are level
> choices and a notion of level.

Not silly at all: if the default level is OK for everyone, there's no
need for the notion of "levels".

> No one disagrees that the default level should provide minimal fontification.

I do.  And many others as well.  That's why the default is and has
always been to use the highest level there is.  And even with this
default, gnu.emacs.help was full (for several years, don't know about
recent cases) of recommendations to add (setq
font-lock-maximum-decoration t) to the user's .emacs.

> The point is to allow users to move to a higher level if they so wish.

The other way around.

>> - "level" is the wrong way to think about this.  Instead, we 
>> should have separate controls for different font-lock features.

> Be specific.  If you are agreeing that users should have choice and
> control when it comes to the degree of font-locking, and you just
> disagree with the current "level" mechanism, then propose
> something specific.

That's exactly what the above does: use separate controls (e.g. bool
vars) for different features.

> Note that in the case I cited the user had the ability to fine-tune
> fontification, for example by customizing individual faces.  But he
> wanted a coarse-grain control, to change the _level_.

I don't know that case, so can't judge why he wanted to change the
level, nor why he wanted this control to be coarse.

The notion of level is wrong, because there are different ways to
characterize the complexity of fontification.  E.g. one of them is the
amount of work done to determine how to highlight the text, another is
the number of distinct elements.  The two aren't necessarily connected
(I almost always want my highlighting to be as precise as possible, but
I generally only want very few elements to be highlighted).

BTW, from what you say, the notion of level was not needed for his
problem since he could get the same result by tweaking faces.
Now tweaking faces on a per-mode basis is not always easy, so we may
need to improve this, but at least this case doesn't seem to be one that
justifies the necessity of levels.


        Stefan

bug#6935: 24.0.50; doc for `font-lock-maximum-decoration'

[Drew Adams]

> >> we should work harder to make sure that the default level is OK
> >> for everyone.
> 
> > That's silly.  If there is a _default_ level then there are level
> > choices and a notion of level.
> 
> Not silly at all: if the default level is OK for everyone, there's no
> need for the notion of "levels".

If the default level is OK to everyone as a _default_ level, that does not imply
that that level is OK for everyone for their individual use.

As one member of everyone, I'm OK with a very minimal level of fontification as
the default for emacs-lisp-mode, but I'm not OK with using that level for
myself.  Being OK with having level X as the default is not the same as being OK
with using level X.

And no, everyone does not agree about which fontification level/degree/feature
_they_ should use.  One size does not fit all.

> > No one disagrees that the default level should provide 
> > minimal fontification.
> 
> I do.  And many others as well.  That's why the default is and has
> always been to use the highest level there is.  And even with this
> default, gnu.emacs.help was full (for several years, don't know about
> recent cases) of recommendations to add (setq
> font-lock-maximum-decoration t) to the user's .emacs.

Dunno whether you are right about what the default has been.  Typically, Emacs
-Q default settings have been minimal in angry-fruit-salad effects, but you
might be right wrt font-lock levels.

Irregardless of what the default values have been, the ability for users to set
the level they want is what you have put in question.  That is where the
disagreement is.

> > The point is to allow users to move to a higher level if 
> > they so wish.
> 
> The other way around.

Either way.  The point is to allow users a choice of level.

But apparently you are saying that the point is to allow users to move to a
_lower_ level if they so wish.  We can agree on that then.  Users deserve to
choose the level they want.  If the default is high, as you say, then they
should be able to choose lower, as you (apparently) claim.

But you apparently disagree with yourself in that case, since you argue both for
letting them move to a lower level and not letting them change level at all (no
levels).

> >> - "level" is the wrong way to think about this.  Instead, we 
> >> should have separate controls for different font-lock features.
> 
> > Be specific.  If you are agreeing that users should have choice and
> > control when it comes to the degree of font-locking, and you just
> > disagree with the current "level" mechanism, then propose
> > something specific.
> 
> That's exactly what the above does: use separate controls (e.g. bool
> vars) for different features.

Be specific.  Which different font-lock features for which mode?  You're just
hand-waving, saying that we could split fontification into a set of "features"
rather than a set of levels.  Sounds fine at that level of abstraction (simply
replacing numeric "levels" by boolean "features"), but the proof is in the
pudding.

> > Note that in the case I cited the user had the ability to fine-tune
> > fontification, for example by customizing individual faces.  But he
> > wanted a coarse-grain control, to change the _level_.
> 
> I don't know that case, so can't judge why he wanted to change the
> level, nor why he wanted this control to be coarse.
> 
> The notion of level is wrong, because there are different ways to
> characterize the complexity of fontification.  E.g. one of them is the
> amount of work done to determine how to highlight the text, another is
> the number of distinct elements.

Another is the visual effect for the user: how much is highlighted, and what is
highlighted or not.

> The two aren't necessarily connected
> (I almost always want my highlighting to be as precise as 
> possible, but I generally only want very few elements to be highlighted).

Sure, there are lots of such considerations.  I don't oppose a superior design
that gives users _more_ control over what gets highlighted, where, how much,
etc.

But where's the beef?  Where's the specific proposal?  Don't just say we should
drop the user control we do offer without offering something better.  If you
give users more control, great.  And not just more fine-grained control, but the
ability to easily chunk that the way they want (into features, levels or
whatever).

> BTW, from what you say, the notion of level was not needed for his
> problem since he could get the same result by tweaking faces.
> Now tweaking faces on a per-mode basis is not always easy, so we may
> need to improve this, but at least this case doesn't seem to 
> be one that justifies the necessity of levels.

Justify the necessity?  Don't be silly.  Emacs itself, and its faces,
fontifications, etc. are _not necessary_.

Changing the level can turn off (or on) lots of regexp processing and the
corresponding use of lots of faces - in this case faces that are used only by
one level and not the other.

Without this separation of regexp processing into two or more groups (levels),
the user would need to customize lots of individual faces to get the effect of
turning off their highlighting.  And that still would not have relieved him of
the processing time for matching their regexps, even if the result were, in
effect, not to highlight any such matches.

A user might want some fontification - some regexps to be matched and their text
highlighted, but not want some other fontification.  However you want to do it
(combine regexp sexps in an easy, customizable way? boolean "features"?
whatever), we should give users the ability to choose (in chunks) what gets
fontified.

As I said earlier, it would be ideal to give users an easy way to define their
own fontification levels or features or groups or whatever.  We're not there
yet.  I'm all in favor of something better than hard-coded levels.  I'm not in
favor of dropping levels in favor of nothing, while ostensibly waiting for pie
in the sky.

bug#6935: 24.0.50; doc for `font-lock-maximum-decoration'

[Stefan Monnier]

> But you apparently disagree with yourself in that case, since you
> argue both for letting them move to a lower level and not letting them
> change level at all (no levels).

Not I do not argue for them to be able to lower the level.  That's just
the functionality currently provided, and which I dislike.

> Be specific.  Which different font-lock features for which mode?
> You're just hand-waving, saying that we could split fontification into
> a set of "features" rather than a set of levels.  Sounds fine at that
> level of abstraction (simply replacing numeric "levels" by boolean
> "features"), but the proof is in the pudding.

I see no need for being more specific: whenever a particular need
arises, we add a corresponding config.

> Sure, there are lots of such considerations.  I don't oppose
> a superior design that gives users _more_ control over what gets
> highlighted, where, how much, etc.

I'm glad we agree.

> But where's the beef?  Where's the specific proposal?  Don't just say
> we should drop the user control we do offer without offering something
> better.

Strawman.


        Stefan

bug#6935: 24.0.50; doc for `font-lock-maximum-decoration'

[Lars Magne Ingebrigtsen]

"Drew Adams" <drew.adams@oracle.com> writes:

> 1. For one thing, the Emacs manual deals with it only using a `setq'
> example:
>
>   (setq font-lock-maximum-decoration
>         '((c-mode . 1) (c++-mode . 1)))
>
> We should tell users how they can use Customize for customizing it.
> (No, it is not obvious how to do that.)  We should not be privileging
> Lisp code in .emacs this way - especially fairly complex Lisp code.

For examples of complex variables like this, I find Lisp code a lot
clearer than convoluted Customize settings.  So this is not a bug, in my
opinion. 

> 2. The doc string and the Customize help for it (same thing) do not help
> much either.  In particular, they are missing the info that if you add
> an entry for one or more modes, then you will likely want to also add
> a catch-all entry for all other modes.

The doc string seems to have been fixed in this regard, with a pretty
clear example.

> 3. Also, to understand what the choice of `level' means here, users need
> to know about fontification levels.  At least a minimum of info about
> that needs to be presented in the Customize (= doc string) help.

I've now mentioned that higher levels mean more fontification.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

bug#6935: 24.0.50; doc for `font-lock-maximum-decoration'

[Drew Adams]

> > the Emacs manual deals with it only using a `setq' example:
> >   (setq font-lock-maximum-decoration
> >         '((c-mode . 1) (c++-mode . 1)))
> >
> > We should tell users how they can use Customize for customizing it.
> > (No, it is not obvious how to do that.)  We should not be 
> > privileging Lisp code in .emacs this way - especially fairly
> > complex Lisp code.
> 
> For examples of complex variables like this, I find Lisp code a lot
> clearer than convoluted Customize settings.  So this is not a 
> bug, in my opinion.

It's not about your personal opinion of Customize.  It's about Emacs's policy of
privileging Customize in user doc.

AFAIK, Emacs Dev _wants_ users to go first to Customize, in general - especially
new users.  Among other things, Customize provides various safety and sanity
checks.

If you have specific improvements in mind for Customize, feel free to submit
them as enhancement requests.  That's unrelated to this bug report.

But the policy has been, in general, to move old suggestions about using `setq'
etc. in .emacs to suggestions about using Customize.

--

FWIW, I too used to resist Customize and do everything using hand-written Lisp
in .emacs.  And I too still think the Customize UI could use a lot of
improvement (to put it politely).

But I finally switched a few years back to using Customize and a separate
`custom-file' for most, and especially for run-of-the-mill, customizations.  I
use Lisp code for things that Customize cannot do, not just to set an option's
value.

bug#6935: 24.0.50; doc for `font-lock-maximum-decoration'

[Chong Yidong]

Maybe we should obsolete font-lock-maximum-decoration in 24.2.  The
concept of decoration levels hasn't been very useful, and removing this
user option is a good first step to get rid of it.

bug#6935: 24.0.50; doc for `font-lock-maximum-decoration'

[Lars Magne Ingebrigtsen]

Chong Yidong <cyd@stupidchicken.com> writes:

> Maybe we should obsolete font-lock-maximum-decoration in 24.2.  The
> concept of decoration levels hasn't been very useful, and removing this
> user option is a good first step to get rid of it.

Sounds like a good idea to me.  I'll mark this report as "pending".

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

bug#6935: 24.0.50; doc for `font-lock-maximum-decoration'

[Drew Adams]

> > Maybe we should obsolete font-lock-maximum-decoration in 24.2.  The
> > concept of decoration levels hasn't been very useful, and 
> > removing this user option is a good first step to get rid of it.
> 
> Sounds like a good idea to me.  I'll mark this report as "pending".

No; it is a bad idea - on its own, that is, without some subsitute/compensating
feature.

And since when does a "_maybe_ we should" suggestion get translated
automatically into a "pending" change?  AFAICT, "pending" means that a decision
has already been made - it is not a "maybe".  And this topic has not even been
raised yet for discussion (by emacs devel)!

Simply removing this feature, without providing some compensating feature as
Stefan suggested, reduces user control.

If you want to propose something better than the current feature, something
that, as Stefan suggested, provides users _more_ control, then fine.  Propose it
to emacs-devel for discussion.  But you should not be _removing_ control willy
nilly from users.

Note: Personally, I use maximum font-lock decoration - this is not about my
personal use of Emacs.  It is about giving users control over their Emacs
experience, or rather not reducing their control further.  (It's also about not
redesigning in (only) a bug thread.)

This is a main point of this bug discussion:

da>  Changing the level can turn off (or on) lots of regexp 
da>  processing and the corresponding use of lots of faces -
da>  in this case faces that are used only by one level and
da>  not the other.
da> 
da>  Without this separation of regexp processing into two or
da>  more groups (levels), the user would need to customize
da>  lots of individual faces to get the effect of turning off
da>  their highlighting.  And that still would not have relieved
da>  him of the processing time for matching their regexps, even
da>  if the result were, in effect, not to highlight any such matches.
da> 
da>  A user might want some fontification - some regexps to be 
da>  matched and their text highlighted, but not want some other
da>  fontification.  However you want to do it (combine regexp
da>  sexps in an easy, customizable way? boolean "features"?
da>  whatever), we should give users the ability to choose (in 
da>  chunks) what gets fontified.
da> 
da>  As I said earlier, it would be ideal to give users an easy 
da>  way to define their own fontification levels or features or
da>  groups or whatever.  We're not there yet.  I'm all in favor
da>  of something better than hard-coded levels.  I'm not in
da>  favor of dropping levels in favor of nothing, while 
da>  ostensibly waiting for pie in the sky.

If you want to propose a _better_ way of letting users chunk font-lock
highlighting, please do.  I'm looking forward to it.

It would be better for users to be able to _define_ (not just choose) such
chunking themselves - better than the limited control they have now over the
essentially hard-coded chunks.  But please do not take away all such chunking
and a user's ability to choose the chunking to use.  And certainly do not do so
without some emacs-devel discussion.