font-lock-maximum-decoration and how to make a default font-lock lower than maximal?

font-lock-maximum-decoration and how to make a default font-lock lower than maximal?

[Vitalie Spinu]

Hi, 

As a mode developer how can I set default font-lock level lower than
maximal (aka t)? 

For example, assume you are developing X-mode and some users would like
to have functions + numbers + other stuff highlighted, but you don't
want to activate those by default. So you can add font-lock keyword
levels (2, 3, 4 etc) to accommodate this preference, but want to set the
default to 2.

What is the emacs recommendation for how to cleanly achieve this without
interfering with user level font-lock-maximum-decoration?

It looks to me that emacs should set font-lock-maximum-decoration to a
number instead to `t' by default. Then mode developers can easily decide
what font-lock to activate by default by setting appropriate keywords up
to that level only, and everything else on a higher level.

Thanks, 
Vitalie.

Re: font-lock-maximum-decoration and how to make a default font-lock lower than maximal?

[Eli Zaretskii]

> From: Vitalie Spinu <spinuvit@gmail.com>
> Date: Fri, 17 Aug 2012 14:29:12 +0200
> 
> It looks to me that emacs should set font-lock-maximum-decoration to a
> number instead to `t' by default. Then mode developers can easily decide
> what font-lock to activate by default by setting appropriate keywords up
> to that level only, and everything else on a higher level.

I think it's not up to the mode developer to decide that.  It's up to
the user.  So you as developer should provide all the fontifications
your users may wish to have, and let them decide what to activate.

Re: font-lock-maximum-decoration and how to make a default font-lock lower than maximal?

[Vitalie Spinu]

  >> Eli Zaretskii <eliz@gnu.org>
  >> on Fri, 17 Aug 2012 17:29:14 +0300 wrote:

  >> From: Vitalie Spinu <spinuvit@gmail.com>
  >> Date: Fri, 17 Aug 2012 14:29:12 +0200
  >> 
  >> It looks to me that emacs should set font-lock-maximum-decoration to a
  >> number instead to `t' by default. Then mode developers can easily decide
  >> what font-lock to activate by default by setting appropriate keywords up
  >> to that level only, and everything else on a higher level.

  > I think it's not up to the mode developer to decide that.  It's up to
  > the user.  So you as developer should provide all the fontifications
  > your users may wish to have, and let them decide what to activate.

You mean "let them decide what to deactivate"? All levels are activated
by default (font-lock-maximum-decoration is t). And obviously some guys
(may be including the developer) don't like that.

Vitalie.

RE: font-lock-maximum-decoration and how to make a defaultfont-lock lower than maximal?

[Drew Adams]

>   > I think it's not up to the mode developer to decide that. 
>   > It's up to the user.  So you as developer should provide
>   > all the fontifications your users may wish to have,

Or that you might want your users to have. ;-)

>   > and let them decide what to activate.
> 
> You mean "let them decide what to deactivate"?

I can't speak for what Eli meant, but yes, let the users decide.

> All levels are activated by default (font-lock-maximum-decoration is t).

Yes, Emacs Dev concluded after many years that `t' is the most appropriate
default value.

As with any user option, one size might not fit all, and it might not be obvious
to users that there is an option for changing this, and even for changing it on
a per-mode basis.  You might want to remind them, for your mode.

> And obviously some guys (may be including the developer) don't like that.

If you mean that you don't like `t' to be the default, then argue here for a
change in default value.  If you mean that you don't like `t' for your own use,
then customize the option.

If you mean that you don't want the default behavior for a mode you create to be
maximum, then you are out of luck.

You can restrict extra fontification in your mode by simply not providing it.
But if you provide more than one fontification level then users get to choose
which level to use, and the default is maximum fontification.

You can also add to the Commentary for your mode (or even to the mode's doc
string, if you think it is important) that you recommend setting the font-lock
decoration for this mode to ___.  Or just remind users about the possibility -
e.g.,

;;  If you want a minimum fontification for Foobar mode,
;;  then customize option `font-lock-maximum-decoration'.
;;  If you want a different fontification level for Foobar
;;  than for other modes, you can do this too by customizing
;;  `font-lock-maximize-decoration'.

Re: font-lock-maximum-decoration should be 2 by default?

[Vitalie Spinu]

  >> "Drew Adams" <drew.adams@oracle.com>
  >> on Fri, 17 Aug 2012 08:57:28 -0700 wrote:

  >> And obviously some guys (may be including the developer) don't like that.

  > If you mean that you don't like `t' to be the default, then argue here for a
  > change in default value.

  > If you mean that you don't want the default behavior for a mode you create to be
  > maximum, then you are out of luck.

I meant both. 

So let me try to defend the new  default:


1) Setting font-lock-maximum-decoration to 2 (i.e. 3 levels recommended
  by emacs info page "23.6.5 Levels of Font Lock") won't impact most of
  the modes, if any at all. Which mode is defining more than 3
  recommended levels?

2) People tend to stick to defaults even when their "true" preferences
  are different. Probably the most dramatic case of life and death is
  the following http://danariely.com/2008/05/05/3-main-lessons-of-psychology/
  (first story).

  In font-lock language: If you design a feature which is intended for
  30% of salad lovers. Then by the virtue of emacs defaults and peoples'
  psychology, 90% of the people will end up using it. That is, 60% of
  normal users (which don't like salads) will end up eating it.

3) Developers which would like to capture 30% of salad lovers will try
  to find workarounds. That is, add redundant, mode-specific font-lock
  customization, or mess with font-lock-maximum-decoration.

4) If not self-obvious, the proposed modification would allow a default
  level of fontification. Thing which is not possible right now.

5) The most knowledgeable person to decide on the default level of font
  lock, is the developer of the mode. Why then emacs would decide that
  the maximum decoration is the best one?


  >> All levels are activated by default (font-lock-maximum-decoration is t).

  > Yes, Emacs Dev concluded after many years that `t' is the most appropriate
  > default value.

Interesting. I am curious of reasons for that. Cannot see one by myself,
but I must be missing something.

Vitalie.

Re: font-lock-maximum-decoration should be 2 by default?

[Eli Zaretskii]

> From: Vitalie Spinu <spinuvit@gmail.com>
> Cc: "'Eli Zaretskii'" <eliz@gnu.org>,  emacs-devel@gnu.org
> Date: Fri, 17 Aug 2012 21:49:04 +0200
> 
>   > Yes, Emacs Dev concluded after many years that `t' is the most appropriate
>   > default value.
> 
> Interesting. I am curious of reasons for that.

Performance.  You don't gain much by omitting some fontifications
nowadays.  The variable was introduced to get snappier redisplay;
since the need for that is no longer valid, the default was set to t.

RE: font-lock-maximum-decoration should be 2 by default?

[Drew Adams]

> > Interesting. I am curious of reasons for that.
> 
> Performance.  You don't gain much by omitting some fontifications
> nowadays.  The variable was introduced to get snappier redisplay;
> since the need for that is no longer valid, the default was set to t.

Do you have any proof of that being _the_ reason it was introduced: just for
performance?

FWIW, that would suggest that the default was initially something other than
`t', which contradicts what Stefan has conjectured.  (I too would guess that the
original default value might not have been `t', but I do not recall.)

And when you say "is no longer valid", apparently you mean that at least since
Emacs 20 it has not been valid, since as you have said elsewhere, 20.1 is when
it became `t'.

I would be surprised if the ability for users to pick the highlighting level
they wanted - with whatever motivation - was not a reason for introducing this
variable.  IOW, my guess is that it has always been about letting users get the
amount of highlighting they want - for whatever reasons, not necessarily for
better redisplay performance.

That some user on a slow machine, long ago, might have chosen minimal
highlighting in part because of performance in no way supports a view that that
is the only reason for a user to choose less highlighting, and it in no way
supports the view that the variable was added only because of slow redisplay.

The variable, now as in the past, gives users control over the amount of
highlighting font lock provides.  Nothing more or less than that.

Yes, that user control is hidden to some extent by the default value being `t',
and to some extent it is stymied by the scarcity of libraries that provide
multiple levels.  Still, the variable lets users choose.

Re: font-lock-maximum-decoration should be 2 by default?

[Eli Zaretskii]

> From: "Drew Adams" <drew.adams@oracle.com>
> Cc: <emacs-devel@gnu.org>
> Date: Fri, 17 Aug 2012 13:53:44 -0700
> 
> > > Interesting. I am curious of reasons for that.
> > 
> > Performance.  You don't gain much by omitting some fontifications
> > nowadays.  The variable was introduced to get snappier redisplay;
> > since the need for that is no longer valid, the default was set to t.
> 
> Do you have any proof of that being _the_ reason it was introduced: just for
> performance?

No proof, just my faulty memory.

Re: font-lock-maximum-decoration should be 2 by default?

[Jason Rumney]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: "Drew Adams" <drew.adams@oracle.com>
>> Cc: <emacs-devel@gnu.org>
>> Date: Fri, 17 Aug 2012 13:53:44 -0700
>> 
>> > > Interesting. I am curious of reasons for that.
>> > 
>> > Performance.  You don't gain much by omitting some fontifications
>> > nowadays.  The variable was introduced to get snappier redisplay;
>> > since the need for that is no longer valid, the default was set to t.
>> 
>> Do you have any proof of that being _the_ reason it was introduced: just for
>> performance?
>
> No proof, just my faulty memory.

In this case, not faulty:

*** lisp/font-lock.el	1997-02-21 09:36:18 +0000
--- lisp/font-lock.el	1997-03-15 13:19:12 +0000
***************
*** 190,196 ****
  If a number, only buffers greater than this size have fontification messages.")
  
  ;;;###autoload
! (defvar font-lock-maximum-decoration nil
    "*Maximum decoration level for fontification.
  If nil, use the default decoration (typically the minimum available).
  If t, use the maximum decoration available.
--- 190,196 ----
  If a number, only buffers greater than this size have fontification messages.")
  
  ;;;###autoload
! (defvar font-lock-maximum-decoration t
    "*Maximum decoration level for fontification.
  If nil, use the default decoration (typically the minimum available).
  If t, use the maximum decoration available.
***************

1996-08-11  Richard Stallman  <rms@psilocin.gnu.ai.mit.edu>

	* Version 19.33 released.

1997-09-15  Richard Stallman  <rms@psilocin.gnu.ai.mit.edu>

	* Version 20.1 released.

(I thought there was also a 19.34, but that might be my faulty memory,
or maybe it was on a branch, so doesn't show up in the changelogs).

RE: font-lock-maximum-decoration should be 2 by default?

[Drew Adams]

> >> > > Interesting. I am curious of reasons for that.
> >> > 
> >> > Performance.  You don't gain much by omitting some fontifications
> >> > nowadays.  The variable was introduced to get snappier redisplay;
> >> > since the need for that is no longer valid, the default 
> >> > was set to t.
> >> 
> >> Do you have any proof of that being _the_ reason it was 
> >> introduced: just for performance?
> >
> > No proof, just my faulty memory.
> 
> In this case, not faulty:
> 
> ! (defvar font-lock-maximum-decoration nil
...
> ! (defvar font-lock-maximum-decoration t
...
> 1996-08-11  Richard Stallman  <rms@psilocin.gnu.ai.mit.edu>
> 	* Version 19.33 released.
> 1997-09-15  Richard Stallman  <rms@psilocin.gnu.ai.mit.edu>
> 	* Version 20.1 released.
> 
> (I thought there was also a 19.34, but that might be my faulty memory,
> or maybe it was on a branch, so doesn't show up in the changelogs).

Uh, where do you see ANYTHING there that supports the claim that "the" reason
why the variable was _introduced_ in the first place was "to get snappier
display"?

What you show supports the fact that the default value was changed from nil to t
in 1997.  Nothing more, AFAICT.  The question is whether the variable and its
levels exist _only_ to provide "snappier redisplay" (by choosing a lower level).

The variable is intended to give users a choice of different amounts of
font-lock decoration.  A user can want more or less decoration for any number of
reasons.  In the old days performance was no doubt one of the possible reasons.

If performance were the only reason for this variable, then those who want to
remove the variable now might have an argument.  (And in that case it could even
have been removed in 1997.)

It is clear to me that there are users, today, who want less fontification in
some modes (I've heard from some of them).  This variable gives them a way to
get that.

Yes, a default of `t' hides this possibility from some users, no doubt.  And
yes, there are too few modes that actually provide different levels.

And yes, one could perhaps dream up a better approach than levels.  But while
waiting for the better approach (conjectured by Stefan long ago), this variable
has the merit of providing users some control over the amount of decoration.

Re: font-lock-maximum-decoration should be 2 by default?

[Jason Rumney]

"Drew Adams" <drew.adams@oracle.com> writes:

> Uh, where do you see ANYTHING there that supports the claim that "the" reason
> why the variable was _introduced_ in the first place was "to get snappier
> display"?

Sorry, I have to rely on memory for that.  I don't know if there is
still an archive of emacs-hackers around on the GNU machines, but it
seems that is what it will take to convince you that Eli and my memories
are correct.

To be more precise, it was introduced originally (in 1995) to allow more
processor intensive font-locking to be computed, without degrading the
default experience for users on older machines. By 1997, those older
machines were scarce enough that the default could be safely changed.

To have this suddenly become an issue in 2012 is somewhat of a surprise.

> It is clear to me that there are users, today, who want less fontification in
> some modes (I've heard from some of them).  This variable gives them a way to
> get that.

I suspect that those users would be just as well served, if not better
served due to the lack of support in many modes for font-lock levels, by
customizing the font-lock faces.

RE: font-lock-maximum-decoration should be 2 by default?

[Drew Adams]

> > It is clear to me that there are users, today, who want 
> > less fontification in some modes (I've heard from some of them).
> > This variable gives them a way to get that.
> 
> I suspect that those users would be just as well served, if not better
> served due to the lack of support in many modes for font-lock 
> levels, by customizing the font-lock faces.

Are you thinking that font lock uses only the `font-lock-*' faces?  Or do you
(hopefully) mean any faces at all that happen to be used for font locking?

Either way, font-locking is about a particular _use_ of a face.  Customizing
that use is not the same as customizing the face itself - which affects all uses
of it.  

Customizing the highlighting used in emacs-lisp-mode should mean just that.
That is not the same as customizing some or all of the faces that happen to be
used for emacs-lisp-mode font-locking.

Sure, if you want to remove some highlighting in a particular mode (e.g., to
simulate a lower decoration level), you can customize some faces used to
highlight that mode.  E.g., you can remove some or all of a face's attributes,
or just make it inherit from `default' to nullify it.

But customizing the face itself affects that face wherever it is used.  That is
a lousy workaround for customizing the _font-locking_ used in a particular mode.

To the extent that a particular mode defines its own faces and uses no others
for font-locking, yes, you can use such a workaround without affecting other
highlighting elsewhere.  But it's still a lousy workaround.  Customizing faces
to be like `default' in order to remove their visible font-lock effects is a
poster child for "workaround".

Re: font-lock-maximum-decoration should be 2 by default?

[Vitalie Spinu]

  >> Jason Rumney <jasonr@gnu.org>
  >> on Sun, 19 Aug 2012 11:34:15 +0800 wrote:

  > "Drew Adams" <drew.adams@oracle.com> writes:
  >> Uh, where do you see ANYTHING there that supports the claim that "the" reason
  >> why the variable was _introduced_ in the first place was "to get snappier
  >> display"?

  > To have this suddenly become an issue in 2012 is somewhat of a surprise.

Issue in 2012 is unrelated to performance. And for some reason this
thread is drifting away from the original request. 

The request was to set font-lock-maximum-decoration to 2, in order to
give to mode developers a much higher flexibility in choosing the
default highlighting, and providing users with higher optional levels
of fortification.

Setting it to t, or removing altogether, imposes one-side-fits-all
philosophy, which is absurd in the case of such a subjective matter as
font-lock.

Vitalie.
  

Re: font-lock-maximum-decoration should be 2 by default?

[Eli Zaretskii]

> From: "Drew Adams" <drew.adams@oracle.com>
> Cc: <spinuvit@gmail.com>, <emacs-devel@gnu.org>
> Date: Sat, 18 Aug 2012 20:13:27 -0700
> 
> Uh, where do you see ANYTHING there that supports the claim that "the" reason
> why the variable was _introduced_ in the first place was "to get snappier
> display"?

There's at least a hint in this commentary at the beginning of
font-lock.el:

 ;; Fontification for a particular mode may be available in a number of levels
 ;; of decoration.  The higher the level, the more decoration, but the more time
 ;; it takes to fontify.  See the variable `font-lock-maximum-decoration', and
 ;; also the variable `font-lock-maximum-size'.  Support modes for Font Lock
 ;; mode can be used to speed up Font Lock mode.  See `font-lock-support-mode'.

RE: font-lock-maximum-decoration should be 2 by default?

[Drew Adams]

> > Uh, where do you see ANYTHING there that supports the claim 
> > that "the" reason why the variable was _introduced_ in the
> > first place was "to get snappier display"?
> 
> There's at least a hint in this commentary at the beginning of
> font-lock.el:
> 
>  ;; Fontification for a particular mode may be available in a 
>  ;; number of levels of decoration.  The higher the level, the
>  ;; more decoration, but the more time it takes to fontify.
>  ;; See the variable `font-lock-maximum-decoration', and
>  ;; also the variable `font-lock-maximum-size'.  Support 
>  ;; modes for Font Lock mode can be used to speed up Font Lock
>  ;; mode.  See `font-lock-support-mode'.

No, not at all.  That does not support a claim that the only reason this
variable was created was performance.  Let alone the stronger claim that the
only reason for its continued existence is performance.

All that comment says is that more fontification generally means slower
performance, and you can speed up performance by customizing the variable.  No
one disputes that.

It's clear that controlling the amount of fontification also affects the
attendant time to fontify.  That does not mean that there are not other reasons,
besides performance, why users might prefer less fontification.  And it does not
mean that performance was the only reason for creating this variable in the
first place.

An attempt now to remove or neuter this option cannot claim that its only raison
d'etre now, or even the only reason for its initial creation, is/was
performance.  We've see no basis for such a claim.

Re: font-lock-maximum-decoration should be 2 by default?

[Andreas Schwab]

Jason Rumney <jasonr@gnu.org> writes:

> In this case, not faulty:
>
> *** lisp/font-lock.el	1997-02-21 09:36:18 +0000
> --- lisp/font-lock.el	1997-03-15 13:19:12 +0000
> ***************
> *** 190,196 ****
>   If a number, only buffers greater than this size have fontification messages.")
>   
>   ;;;###autoload
> ! (defvar font-lock-maximum-decoration nil
>     "*Maximum decoration level for fontification.
>   If nil, use the default decoration (typically the minimum available).
>   If t, use the maximum decoration available.
> --- 190,196 ----
>   If a number, only buffers greater than this size have fontification messages.")
>   
>   ;;;###autoload
> ! (defvar font-lock-maximum-decoration t
>     "*Maximum decoration level for fontification.
>   If nil, use the default decoration (typically the minimum available).
>   If t, use the maximum decoration available.
> ***************

That looks rather like an accident.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: font-lock-maximum-decoration should be 2 by default?

[Eli Zaretskii]

> From: Andreas Schwab <schwab@linux-m68k.org>
> Cc: Eli Zaretskii <eliz@gnu.org>,  spinuvit@gmail.com,  Drew Adams <drew.adams@oracle.com>,  emacs-devel@gnu.org
> Date: Sun, 19 Aug 2012 12:34:48 +0200
> 
> > *** lisp/font-lock.el	1997-02-21 09:36:18 +0000
> > --- lisp/font-lock.el	1997-03-15 13:19:12 +0000
> > ***************
> > *** 190,196 ****
> >   If a number, only buffers greater than this size have fontification messages.")
> >   
> >   ;;;###autoload
> > ! (defvar font-lock-maximum-decoration nil
> >     "*Maximum decoration level for fontification.
> >   If nil, use the default decoration (typically the minimum available).
> >   If t, use the maximum decoration available.
> > --- 190,196 ----
> >   If a number, only buffers greater than this size have fontification messages.")
> >   
> >   ;;;###autoload
> > ! (defvar font-lock-maximum-decoration t
> >     "*Maximum decoration level for fontification.
> >   If nil, use the default decoration (typically the minimum available).
> >   If t, use the maximum decoration available.
> > ***************
> 
> That looks rather like an accident.

Not an accident:

  1997-03-15  Simon Marshall  <simon@wombat.gnu.ai.mit.edu>

	  * font-lock.el (font-lock-keyword-depth): New function.
	  Use it wherever extra types are used to calculate parenthesis depth.
	  (c-font-lock-extra-types, c++-font-lock-extra-types)
	  (objc-font-lock-extra-types, java-font-lock-extra-types)
	  (font-lock-keywords): Doc fix.
	  (font-lock-defaults, font-lock-defaults-alist): Doc switch.
	  (font-lock-maximum-decoration): Default to t.
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

RE: font-lock-maximum-decoration should be 2 by default?

[Drew Adams]

>   > Yes, Emacs Dev concluded after many years that `t' is the 
>   > most appropriate default value.
> 
> Interesting. I am curious of reasons for that. Cannot see one 
> by myself, but I must be missing something.

Actually, the default has been `t' for as long as I can recall.  Eli has said it
is since 20.1.  Stefan has conjectured that it is ever since the variable has
existed.
http://lists.gnu.org/archive/html/emacs-devel/2007-10/msg00799.html


But these too might be of interest.  Suggestion: start with the last.  This
useful feature (font-lock levels) is apparently already on its way to the trash
heap (its removal is marked `pending' for 24.2).  We will apparently soon have
no choice in the matter - the behavior will be hard-coded as maximum decoration.
http://lists.gnu.org/archive/html/emacs-devel/2011-06/msg00587.html
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=6935#17
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=6935#23
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=6935#47

Re: font-lock-maximum-decoration should be 2 by default?

[Vitalie Spinu]

  >> "Drew Adams" <drew.adams@oracle.com>
  >> on Fri, 17 Aug 2012 13:50:56 -0700 wrote:

  >> > Yes, Emacs Dev concluded after many years that `t' is the 
  >> > most appropriate default value.
  >> 
  >> Interesting. I am curious of reasons for that. Cannot see one 
  >> by myself, but I must be missing something.

  > Actually, the default has been `t' for as long as I can recall.  Eli has said it
  > is since 20.1.  Stefan has conjectured that it is ever since the variable has
  > existed.
  > http://lists.gnu.org/archive/html/emacs-devel/2007-10/msg00799.html

Apparently it is t from 1997 http://article.gmane.org/gmane.emacs.devel/66833/match=font+lock+maximum+decoration

  > But these too might be of interest.  Suggestion: start with the last.  This
  > useful feature (font-lock levels) is apparently already on its way to the trash
  > heap (its removal is marked `pending' for 24.2).  We will apparently soon have
  > no choice in the matter - the behavior will be hard-coded as maximum
  > decoration.

That's funny. Probably the most subjective part of Emacs, fontification,
is left without even basic customization?  

If that happens, every mode will try to reinvent its own font-lock
levels and implement its own user settings.  IMHO this is chaos and
clearly a regression.


Most of people agree on the basic fontification. Trouble comes for
higher levels. Could it be please reset to the default of 2?

Re: font-lock-maximum-decoration should be 2 by default?

[Eli Zaretskii]

> From: Vitalie Spinu <spinuvit@gmail.com>
> Cc: "'Eli Zaretskii'" <eliz@gnu.org>,  <emacs-devel@gnu.org>
> Date: Sat, 18 Aug 2012 00:47:02 +0200
> 
> That's funny. Probably the most subjective part of Emacs, fontification,
> is left without even basic customization?  
> 
> If that happens, every mode will try to reinvent its own font-lock
> levels and implement its own user settings.  IMHO this is chaos and
> clearly a regression.
> 
> 
> Most of people agree on the basic fontification. Trouble comes for
> higher levels.

That's not my experience.  In my experience, most people want the
fontifications always be ON, and there's a small minority that doesn't
want _any_ fontifications at all (which one gets by turning
global-font-lock-mode OFF).

Re: font-lock-maximum-decoration should be 2 by default?

[Vitalie Spinu]

  >> Vitalie Spinu <spinuvit@gmail.com>
  >> on Sat, 18 Aug 2012 00:47:02 +0200 wrote:

  >> "Drew Adams" <drew.adams@oracle.com>
  >> on Fri, 17 Aug 2012 13:50:56 -0700 wrote:

  >> > Yes, Emacs Dev concluded after many years that `t' is the 
  >> > most appropriate default value.
  >> 
  >> Interesting. I am curious of reasons for that. Cannot see one 
  >> by myself, but I must be missing something.

  >> Actually, the default has been `t' for as long as I can recall.  Eli has said it
  >> is since 20.1.  Stefan has conjectured that it is ever since the variable has
  >> existed.
  >> http://lists.gnu.org/archive/html/emacs-devel/2007-10/msg00799.html

  > Apparently it is t from 1997 http://article.gmane.org/gmane.emacs.devel/66833/match=font+lock+maximum+decoration

  >> But these too might be of interest.  Suggestion: start with the last.  This
  >> useful feature (font-lock levels) is apparently already on its way to the trash
  >> heap (its removal is marked `pending' for 24.2).  We will apparently soon have
  >> no choice in the matter - the behavior will be hard-coded as maximum
  >> decoration.

  > That's funny. Probably the most subjective part of Emacs, fontification,
  > is left without even basic customization?


How about the following as a replacement for levels:

Let say you are defining mode "X". Then `define-derived-mode' and
`define-generic-mode' can add X-mode-font-lock-keywords as a
(customizable?) variable, which would be a list of keywords or symbols
holding keywords.

Then font-lock can bunch together font-lock-keywords and
X-mode-font-lock-keywords at initialization. 

Each mode can define a set of keyword variables with suggestive names
X-mode-fl-keyword-basic, X-mode-fl-keyword-functions,
X-mode-fl-keyword-brackets, or just X-mode-fl-keyword-1 for
level-1. Emacs can define a bunch of standard keywords as well.

Then user will be able to add/remove keywords from
X-mode-font-lock-keywords as desired.

Vitalie.

Re: font-lock-maximum-decoration should be 2 by default?

[Stefan Monnier]

> Let say you are defining mode "X". Then `define-derived-mode' and
> `define-generic-mode'

[ I dislike define-generic-mode so I'm opposed to improving it. ]

> can add X-mode-font-lock-keywords as a (customizable?) variable, which
> would be a list of keywords or symbols holding keywords.

That might be a good way to replace the ill-defined notion of levels, yes.


        Stefan

Re: font-lock-maximum-decoration should be 2 by default?

[Vitalie Spinu]

  >> Stefan Monnier <monnier@IRO.UMontreal.CA>
  >> on Tue, 21 Aug 2012 13:31:42 -0400 wrote:

  >> Let say you are defining mode "X". Then `define-derived-mode' and
  >> `define-generic-mode'

  > [ I dislike define-generic-mode so I'm opposed to improving it. ]

  >> can add X-mode-font-lock-keywords as a (customizable?) variable, which
  >> would be a list of keywords or symbols holding keywords.

  > That might be a good way to replace the ill-defined notion of levels, yes.

Good! Then I will wrap a preliminary version of this feature for further
discussion.

Vitalie.

RE: font-lock-maximum-decoration should be 2 by default?

[Drew Adams]

> > Let say you are defining mode "X". Then `define-derived-mode' and
> > `define-generic-mode' can add X-mode-font-lock-keywords as a
> > (customizable?) variable, which would be a list of keywords or
> > symbols holding keywords.
> 
> That might be a good way to replace the ill-defined notion of 
> levels, yes.

It's not about a "notion".  The name "levels" is a misnomer, but the mechanism
of easily defining different sets/kinds of highlighting for a given mode, and
interactively switching among them, is a plus, not a minus.

Just providing `X-mode-font-lock-keywords' for mode X does not replace such a
mechanism, at all.

Letting users customize multiple such sets of highlighting patterns for a given
mode is a start, but what's also needed is an easy way to interactively switch
among them.

Today's misnamed "levels" are such sets of mode-specific highlighting patterns
(choices).  And you can easily switch among them in a given mode.

We should not be throwing away that possibility just because Emacs Dev has not
kept up with predefining such sets for various modes.  Make it easier for users
do define them and easier for users to switch among them.  _That_ would be an
improvement.

Re: font-lock-maximum-decoration should be 2 by default?

[Stephen J. Turnbull]

Vitalie Spinu writes:

 >   In font-lock language: If you design a feature which is intended for
 >   30% of salad lovers. Then by the virtue of emacs defaults and peoples'
 >   psychology, 90% of the people will end up using it. That is, 60% of
 >   normal users (which don't like salads) will end up eating it.

True, but it's not clear that Emacs should care about "normal" users
in the sense of "people's psychology".  Emacs users are different, at
least that's the conventional wisdom.  They like (1) customizability,
(2) a consistent user interface across applications.  It's not obvious
that the generally prevalent "accept the default" psychology is that
relevant to Emacs users.

 > 3) Developers which would like to capture 30% of salad lovers will try
 >   to find workarounds. That is, add redundant, mode-specific font-lock
 >   customization, or mess with font-lock-maximum-decoration.

This is true, but I'm not sure if it's a problem.

 > 4) If not self-obvious, the proposed modification would allow a default
 >   level of fontification. Thing which is not possible right now.

It's not obvious that the concept of "level of fontification" is
entirely consistent.  At least for me, if certain features aren't
fontified, I'm unhappy with the fontification, and if others are, I
get annoyed.  If those sets are basically monotonic across most Emacs
users, you can talk about levels.  Otherwise, what you mean by "level"
is what you mean, no more and no less.

 > 5) The most knowledgeable person to decide on the default level of font
 >   lock, is the developer of the mode. Why then emacs would decide that
 >   the maximum decoration is the best one?

This is not at all obvious.  In principle, the most knowledgeable
person to decide on the level of fontification is the user.  Then the
best default is the one that best serves the most users.  But "best
serve" is not well-defined, either.  One might think that the defaults
that require the fewest customizations by the typical user are best,
but it could easily be the case that the best defaults somehow
demonstrate the capabilities of Emacs, allowing users to learn them
with little effort, and so enabling the users to choose the ones they
like with less effort.

Whatever it is that "typical" users want, it's best decided by those
who observe the typical users.  If a mode is used by a wide variety of
users, then the Emacs maintainers are probably in the best position to
observe "typical" needs.  If a mode is used mostly by a subset of
"activists" (== users who frequently interact with the mode
maintainers) then the mode maintainers probably know best.  (Yet even
here there is a caveat: the Emacs maintainers seem likely to be most
knowledgeable about how to introduce modes to new users not already in
the activist sets.)

In the end there are arguments for both sides.

Re: font-lock-maximum-decoration should be 2 by default?

[Vitalie Spinu]

  >> "Stephen J. Turnbull" <stephen@xemacs.org>
  >> on Sat, 18 Aug 2012 14:10:58 +0900 wrote:

  > Vitalie Spinu writes:
  >> In font-lock language: If you design a feature which is intended for
  >> 30% of salad lovers. Then by the virtue of emacs defaults and peoples'
  >> psychology, 90% of the people will end up using it. That is, 60% of
  >> normal users (which don't like salads) will end up eating it.

  > True, but it's not clear that Emacs should care about "normal" users
  > in the sense of "people's psychology".  Emacs users are different, at
  > least that's the conventional wisdom.  They like (1) customizability,
  > (2) a consistent user interface across applications.  It's not obvious
  > that the generally prevalent "accept the default" psychology is that
  > relevant to Emacs users.

It's easy to get sick of too much customization. It's another well know
paradox of human pshychology -- we want more choose but too much choice
is bad for you
(http://news.bbc.co.uk/today/hi/today/newsid_8155000/8155505.stm).

There are so many small inconveniences/bugs which I know I can solve
probably in 15-30 minutes by studding the code/docs/customization, but I
continue to leave with those in emacs, sometimes for months and
years. Familiar?

  >> 3) Developers which would like to capture 30% of salad lovers will try
  >> to find workarounds. That is, add redundant, mode-specific font-lock
  >> customization, or mess with font-lock-maximum-decoration.

  > This is true, but I'm not sure if it's a problem.

It's a problem in light of yours (2). Everyone wants a consistent
interface.

  >> 4) If not self-obvious, the proposed modification would allow a default
  >> level of fontification. Thing which is not possible right now.

  > It's not obvious that the concept of "level of fontification" is
  > entirely consistent.  At least for me, if certain features aren't
  > fontified,

I agree, levels are not flexible enough (or at least at higher
levels). People tend to agree on the basic fontification like strings,
comments and keywords. But with more fontification levels become a
trouble. For example I can choose to fortify the function call as in
"foo(x, y)" or I can choose to fortify parenthesis. Different people
might choose different things. Also, I might want to fortify {} braces
as they are difficult to distinguish from (), but leave all other paren
syntax untouched.  These things are difficult or impossible to fit into
levels.

Vitalie.

Re: font-lock-maximum-decoration should be 2 by default?

[Stephen J. Turnbull]

Vitalie Spinu writes:

 > It's easy to get sick of too much customization. It's another well know
 > paradox of human pshychology -- we want more choose but too much choice
 > is bad for you

And the experiments you cite were conducted on a sample of Emacs
users?  Surely not!  The point is that Emacs users *are* different
(this is nearly a tautology), and those in the best position to judge
*how* different are the maintainers.

 > There are so many small inconveniences/bugs which I know I can solve
 > probably in 15-30 minutes by studding the code/docs/customization, but I
 > continue to leave with those in emacs, sometimes for months and
 > years. Familiar?

Yes, with any program by Microsoft and most of the software on my
iPhone.  Not in the context of Emacs.  It took about two hours to
establish the habit of dealing with minor customizations immediately
and quite a bit longer (a couple of months) to establish the habit of
recording long activities I might want to repeat verbatim and writing
quick scripts for those that are basically repetitive but need
variation on repetition.  These small investments in Emacs have been
repaid many times.[1]  But they are impossible with most programs.

I know I'm at one extreme of the spectrum; at least some Emacs users
use it because it was the only decent editor available on their host,
or because the best editor for their application was an Emacs mode,
and everything else they just don't want to deal with but use Emacs
"as is" just because it's there.  Nevertheless, that extreme *is*
achieved (I'm one example, and I'm not the only one), and I suspect a
large share of Emacs users tend to this end of the spectrum.

 >   >> 3) Developers which would like to capture 30% of salad lovers will try
 >   >> to find workarounds. That is, add redundant, mode-specific
 >   >> font-lock customization, or mess with
 >   >> font-lock-maximum-decoration.
 > 
 >   > This is true, but I'm not sure if it's a problem.
 > 
 > It's a problem in light of yours (2). Everyone wants a consistent
 > interface.

But who decides what "consistent" means?  The highlighting appropriate
for C is surely different from that for Lisp or Perl or Haskell.  Some
will be common, I suppose, such as for literal strings.  But what
about Perl "barewords"?  What does consistency mean for something that
doesn't exist in most languages?  Should shell commands be highlighted
differently depending on whether they're builtins, system commands, or
shell functions and aliases?

That kind of thing has to be decided by the mode developers, although
the Emacs maintainers often will offer suggestions, and for modes in
core Emacs will add/adjust to their own taste.  And in the end the
dirge "Why do the defaults always suck?" applies anyway.

Footnotes: 
[1]  And then I needed to break them when I started reviewing others'
requirement suggestions and code contributions -- reviewers need to
make some decisions about defaults, and adjusting to personal taste
because your initial reaction is "this SUCKS!!" is inappropriate --
but that's a different story.

Re: font-lock-maximum-decoration should be 2 by default?

[Vitalie Spinu]

  >> "Stephen J. Turnbull" <stephen@xemacs.org>
  >> on Sun, 19 Aug 2012 20:10:21 +0900 wrote:

  > Vitalie Spinu writes:
  >> It's easy to get sick of too much customization. It's another well know
  >> paradox of human pshychology -- we want more choose but too much choice
  >> is bad for you

  > And the experiments you cite were conducted on a sample of Emacs
  > users?  Surely not!  The point is that Emacs users *are* different
  > (this is nearly a tautology), 

 The full spectrum of Emacs users is broader than you seem to
 think. It's not only hardcore programmers, like you are. I met a guy
 who was using Emacs only for org-mode to organize himself! I also won't
 be surprised that the majority of Emacs users don't actually know
 enough elisp to be able to write useful scripts. As one of the
 developers of ESS, I see users which have no clue of how to customize
 even basic things in Emacs. They are still able to happily use
 ESS/Emacs, but they all rely on defaults. And this is why defaults are
 so important. So knowing elisp, or even the ability to customize things
 is not a prerequisite for an Emacs user at all.

  >> >> 3) Developers which would like to capture 30% of salad lovers will try
  >> >> to find workarounds. That is, add redundant, mode-specific
  >> >> font-lock customization, or mess with
  >> >> font-lock-maximum-decoration.
  >> 
  >> > This is true, but I'm not sure if it's a problem.
  >> 
  >> It's a problem in light of yours (2). Everyone wants a consistent
  >> interface.

  > But who decides what "consistent" means?  The highlighting appropriate
  > for C is surely different from that for Lisp or Perl or Haskell.  Some
  > will be common, I suppose, such as for literal strings.  But what
  > about Perl "barewords"?  What does consistency mean for something that
  > doesn't exist in most languages?  Should shell commands be highlighted
  > differently depending on whether they're builtins, system commands, or
  > shell functions and aliases?

Consistent in customization interface, not in that fontification is set
by default. If there is variable font-lock-maximum-decoration that
should be it. No X-mode-maximum-font-lock, foo-mode-best-font-lock or
boo-mode-try-this-font-lock-level etc.

  > That kind of thing has to be decided by the mode developers, although
  > the Emacs maintainers often will offer suggestions, and for modes in
  > core Emacs will add/adjust to their own taste.  And in the end the
  > dirge "Why do the defaults always suck?" applies anyway.

:)

Re: font-lock-maximum-decoration should be 2 by default?

[Stephen J. Turnbull]

Vitalie Spinu writes:

 >  The full spectrum of Emacs users is broader than you seem to
 >  think.

I don't disagree with you on the *range* of the distribution of Emacs
users; I disagree with you on the *location* of the main weight of the
distribution.  Examples speak to the question of range, but I already
gave my own regarding the two extremes.  And as I already pointed out, there are people who
should have better information about location than either of us: the
maintainers.

 > It's not only hardcore programmers, like you are.

I don't consider myself a hardcore programmer.  What makes you think I
am?  I simply have a broad range of interests, and digging into code
is one of them.  Writing code is not something I have much time for on
a regular basis, unfortunately.  My contributions are mostly in the
line of review, documentation, standards-mongering, and release
management.

 > Consistent in customization interface, not in that fontification is
 > set by default. If there is variable font-lock-maximum-decoration
 > that should be it. No X-mode-maximum-font-lock,
 > foo-mode-best-font-lock or boo-mode-try-this-font-lock-level etc.

I agree with that, who wouldn't?  But that's not what you wrote
before, and it doesn't help with the question of setting good
defaults.

Steve

Re: font-lock-maximum-decoration and how to make a default font-lock lower than maximal?

[Eli Zaretskii]

> From: Vitalie Spinu <spinuvit@gmail.com>
> Cc: emacs-devel@gnu.org
> Date: Fri, 17 Aug 2012 16:42:31 +0200
> 
>   > I think it's not up to the mode developer to decide that.  It's up to
>   > the user.  So you as developer should provide all the fontifications
>   > your users may wish to have, and let them decide what to activate.
> 
> You mean "let them decide what to deactivate"?

Yes (but in English this is the same thing).

> All levels are activated by default (font-lock-maximum-decoration is
> t). And obviously some guys (may be including the developer) don't
> like that.

The developer, acting as a user, is free to customize his/her Emacs to
their heart's content.

Re: font-lock-maximum-decoration and how to make a default font-lock lower than maximal?

[Stefan Monnier]

> For example, assume you are developing X-mode and some users would like
> to have functions + numbers + other stuff highlighted, but you don't
> want to activate those by default. So you can add font-lock keyword
> levels (2, 3, 4 etc) to accommodate this preference, but want to set the
> default to 2.

I hate font-lock-maximum-decoration's notion of "levels" because there
is no such neat line.  Some users will care about one particular detail,
others about an other.  So please don't use font-lock's levels for that
fine-grained control.  Instead just introduce new config vars specific
to your major mode.


        Stefan

RE: font-lock-maximum-decoration and how to make a default font-locklower than maximal?

[Drew Adams]

> > For example, assume you are developing X-mode and some 
> > users would like to have functions + numbers + other stuff
> > highlighted, but you don't want to activate those by default.
> > So you can add font-lock keyword levels (2, 3, 4 etc) to
> > accommodate this preference, but want to set the default to 2.
> 
> I hate font-lock-maximum-decoration's notion of "levels" because
> there is no such neat line.  Some users will care about one
> particular detail, others about an other.  So please don't use
> font-lock's levels for that fine-grained control.  Instead just
> introduce new config vars specific to your major mode.

You are throwing out the baby with the bath water.
And you are missing the point about font-lock "levels".

Font-lock levels are not just for users to _customize_ decoration for a
particular mode, in a single, fixed way.

Font-lock levels let users easily (e.g., using a cycle command or picking a
"level" from a menu, as in `font-menus.el' from FJ Wright) switch to a different
kind of highlighting.

Those kinds are called "levels" today, but you need not (and probably should
not) think of them as such.  "Level" here is a misnomer.

So-called "levels" are just different highlighting patterns.  These patterns
might or might not reflect different highlighting degrees ("levels"), however
you might want to imagine such leveling.  There is no need to think in terms of
degree at all.  Think "different", not "more" or "less".

Whether users should be able to easily customize the "levels" available for a
given mode is a good question.  I certainly am all for that possibility.

By all means, let users customize what the font-locking choices are for a given
mode, and how many choices there are.  Go for it.  And if the doc and names used
for this customization emphasize that these are just arbitrary sets of font-lock
appearances, rather than "levels", so much the better.

But please do not take away the ability to interactively switch among such
highlighting choices, which is provided by today's "levels" mechanism (with a
little help from `font-menus.el' etc.).

The facts that (1) most Emacs modes do not currently provide more than one
"level" by default, and that (2) many users are unaware that they can choose a
highlighting pattern ("level") when such a choice does exist, is not a reason
for tossing the mechanism.  (Choose = customize once and for all or switch
patterns interactively.)

Rather, those facts are reasons for Emacs Dev to (a) offer different predefined
highlighting choices for more modes, (b) advertise the feature more, and (c)
provide better interactivity (switching among choices) out of the box.