Indentation of def*

Indentation of def*

[Lars Ingebrigtsen]

A longstanding issue is whether to do anything about this heuristic in
our indentation code:

(defun definitely-regular (a b))
(defun still-regular (a b))

(progn
  (definitely-regular a
    b)
  (still-regular a
                 b))

That is, any function or macro that has a name that starts with "def" is
indented similarly to a `defun' (etc).

This is discussed in bug#43329, and all two of us discussing this have
landed on "let's fix this".  So today I've tagged all in-tree def*
function/macros with correct indentation (ahem), so if we remove the
heuristic from the indentation function, in-tree code won't change how
it's indented.  (By and large -- there's a handful of functions that
will change, but the affected call sites should be less than a dozen, if
I got it right.)

But code that uses out-of-tree functions named def* will see indentation
changes until everybody tags up their functions with

(defun define-whatever ()
  (declare (indent defun))

And people going back and forth between Emacs 29 and earlier versions
will see the indentation change (for untagged functions), which is why
we rarely change how Emacs Lisp is indented.

So this is the thread where you can say "yay" or "eek".

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Indentation of def*

[Stefan Monnier]

> So this is the thread where you can say "yay" or "eek".

I'm in favor if dropping the heuristic.


        Stefan

Re: Indentation of def*

[Po Lu]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> And people going back and forth between Emacs 29 and earlier versions
> will see the indentation change (for untagged functions), which is why
> we rarely change how Emacs Lisp is indented.
>
> So this is the thread where you can say "yay" or "eek".

Will it work to only use defun indentation for where the name begins
with `def' and only contains one word, and for anything that begins with
`define'?

I think the heuristic is too commonly used to be straight out removed.

Re: Indentation of def*

[Lars Ingebrigtsen]

Po Lu <luangruo@yahoo.com> writes:

> Will it work to only use defun indentation for where the name begins
> with `def' and only contains one word, and for anything that begins with
> `define'?

There's `def-edebug-elem-spec', `deftest-make-overlay-1' and
`defcustom-mh', for instance.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Indentation of def*

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Po Lu <luangruo@yahoo.com> writes:
>
>> Will it work to only use defun indentation for where the name begins
>> with `def' and only contains one word, and for anything that begins with
>> `define'?
>
> There's `def-edebug-elem-spec', `deftest-make-overlay-1' and
> `defcustom-mh', for instance.

(And vice versa, there's oodles of functions that have names that start
with "define" that should not be indented as defuns, like
`define-mail-alias'.)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Indentation of def*

[Po Lu]

Lars Ingebrigtsen <larsi@gnus.org> writes:

>> There's `def-edebug-elem-spec', `deftest-make-overlay-1' and
>> `defcustom-mh', for instance.
>
> (And vice versa, there's oodles of functions that have names that start
> with "define" that should not be indented as defuns, like
> `define-mail-alias'.)

But still, I think my solution would cover _most_ of the existing cases,
no?

Thanks.

Re: Indentation of def*

[Lars Ingebrigtsen]

Po Lu <luangruo@yahoo.com> writes:

> But still, I think my solution would cover _most_ of the existing cases,
> no?

We want to move away from heuristics, not tweak them to make them 3%
"better" (and even more incomprehensible).

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Indentation of def*

[Po Lu]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> So this is the thread where you can say "yay" or "eek".

Then In this case, I would have to say "eek".  There is simply too much
code out there (mine included) that relies on the existing heuristic,
and it would be very annoying to have that broken, at least now.

Re: Indentation of def*

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 885 bytes --]

On Thu, Oct 14, 2021, 13:27 Po Lu <luangruo@yahoo.com> wrote:

> Lars Ingebrigtsen <larsi@gnus.org> writes:
>
> > So this is the thread where you can say "yay" or "eek".
>
> Then In this case, I would have to say "eek".  There is simply too much
> code out there (mine included) that relies on the existing heuristic,
> and it would be very annoying to have that broken, at least now.
>

I think I reported that bug originally.  I'm half-yay half-eek. Po Lu's use
case is definitely something to consider.

I'm in favor of tweaking the heuristic and applying it only to macro
definitions, which are (or should be) the majority of definition creating
forms.

I can't confirm right now, but I think SLIME/Sly use a similar heuristic
for Common Lisp.

The plan would be to add the indent spec only to functions that define
things, or to make those macros.

João

[-- Attachment #2: Type: text/html, Size: 1506 bytes --]

Re: Indentation of def*

[Stefan Kangas]

Po Lu <luangruo@yahoo.com> writes:

> Then In this case, I would have to say "eek".  There is simply too much
> code out there (mine included) that relies on the existing heuristic,
> and it would be very annoying to have that broken, at least now.

Can't you just use this?

    (declare (indent defun))

And shouldn't you be using that in any case, to make your intention more
clear, and avoid relying on a fundamentally brittle heuristic?

Re: Indentation of def*

[Po Lu]

Stefan Kangas <stefankangas@gmail.com> writes:

> Can't you just use this?
>
>     (declare (indent defun))

I could, but I was just demonstrating that a large amount of code
depends on the previous behaviour in order to be indented nicely.

> And shouldn't you be using that in any case, to make your intention more
> clear, and avoid relying on a fundamentally brittle heuristic?

If it's a heuristic that worked (at least, AFAIR) in Emacs 18, then I'm
usually inclined to trust it.  (And I considered it smart at the time,
when I first discovered it).

Re: Indentation of def*

[Lars Ingebrigtsen]

João Távora <joaotavora@gmail.com> writes:

> I'm in favor of tweaking the heuristic and applying it only to macro
> definitions, which are (or should be) the majority of definition
> creating forms.

In-tree, there are more 2x as many macros that are called def* than
functions, and of those functions, about half of them really wants to
have (indent defun).  (And most of the macros, but not all.)

So the macro/function thing is significant, but not compelling.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Indentation of def*

[Stefan Monnier]

> Then In this case, I would have to say "eek".  There is simply too much
> code out there (mine included) that relies on the existing heuristic,
> and it would be very annoying to have that broken, at least now.

The upside is that it's trivial to fix!
(the `(declare (indent defun))` form works since Emacs-21, so there's no
backward compatibility issue).


        Stefan

Re: Indentation of def*

[Stefan Kangas]

Po Lu <luangruo@yahoo.com> writes:

> If it's a heuristic that worked (at least, AFAIR) in Emacs 18, then I'm
> usually inclined to trust it.  (And I considered it smart at the time,
> when I first discovered it).

It's clearly not smart enough though, and fails in some cases.

I don't see how you could design a heuristic that won't fail, so better
than emacs-lisp-mode silently trying to fix this in a "smart" way (that
leads to some surprising results) is to force the user to be explicit
about her expectations.  That's a step forward, IMO.

Re: Indentation of def*

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 914 bytes --]

On Thu, Oct 14, 2021 at 2:40 PM Lars Ingebrigtsen <larsi@gnus.org> wrote:

> João Távora <joaotavora@gmail.com> writes:
>
> > I'm in favor of tweaking the heuristic and applying it only to macro
> > definitions, which are (or should be) the majority of definition
> > creating forms.
>
> In-tree, there are more 2x as many macros that are called def* than
> functions, and of those functions, about half of them really wants to
> have (indent defun).  (And most of the macros, but not all.)
>
> So the macro/function thing is significant, but not compelling.


I'm sorry I couldn't parse this.  I sense you're trying to make
some kind of argument backed by statistics, but I cannot
understand it.  Can you explain again in some more detail,
for my benefit?.

Maybe with a short example for each of the classes you
mentioned.  And in-tree counts for each of those classes.

Thanks,
João

[-- Attachment #2: Type: text/html, Size: 1410 bytes --]

Re: Indentation of def*

[Po Lu]

Stefan Kangas <stefankangas@gmail.com> writes:

> leads to some surprising results) is to force the user to be explicit
> about her expectations.  That's a step forward, IMO.

That would be a step backwards, at least without a grace period for all
the code out there to adjust.

Re: Indentation of def*

[Stefan Kangas]

Po Lu <luangruo@yahoo.com> writes:

> That would be a step backwards, at least without a grace period for all
> the code out there to adjust.

I don't think it's a bad idea to announce this change as planned for
Emacs 29.1 in the Emacs 28.1 NEWS file.

Re: Indentation of def*

[Po Lu]

Stefan Kangas <stefankangas@gmail.com> writes:

> I don't think it's a bad idea to announce this change as planned for
> Emacs 29.1 in the Emacs 28.1 NEWS file.

LGTM, thanks.

Re: Indentation of def*

[Eli Zaretskii]

> From: Stefan Kangas <stefankangas@gmail.com>
> Date: Thu, 14 Oct 2021 17:50:18 -0700
> Cc: Lars Ingebrigtsen <larsi@gnus.org>, emacs-devel@gnu.org
> 
> Po Lu <luangruo@yahoo.com> writes:
> 
> > That would be a step backwards, at least without a grace period for all
> > the code out there to adjust.
> 
> I don't think it's a bad idea to announce this change as planned for
> Emacs 29.1 in the Emacs 28.1 NEWS file.

We don't make such announcements in NEWS.  That file describes the
changes actually done in the version with which it comes.  If there
_is_ a change we make in preparation for some future action, then we
can indeed tell that in NEWS.  But not as a "future NEWS" without any
anchor in the current release: that's a slippery slope, because we
might as well change our minds in various ways before the next
release.

Re: Indentation of def*

[Stefan Kangas]

Eli Zaretskii <eliz@gnu.org> writes:

> We don't make such announcements in NEWS.  That file describes the
> changes actually done in the version with which it comes.  If there
> _is_ a change we make in preparation for some future action, then we
> can indeed tell that in NEWS.

I was thinking of something like this line from etc/NEWS.27:

    ** The ftx font backend driver is now obsolete and will be removed in
    Emacs 28.

AFAIK, the above message was just informational and came with no
functional changes.

Re: Indentation of def*

[Lars Ingebrigtsen]

João Távora <joaotavora@gmail.com> writes:

> Maybe with a short example for each of the classes you 
> mentioned.  And in-tree counts for each of those classes.

I don't think so.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Indentation of def*

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 905 bytes --]

On Fri, Oct 15, 2021 at 10:57 AM Lars Ingebrigtsen <larsi@gnus.org> wrote:

> João Távora <joaotavora@gmail.com> writes:
>
> > Maybe with a short example for each of the classes you
> > mentioned.  And in-tree counts for each of those classes.
>
> I don't think so.
>

Surely, there must be a misunderstanding.  I'm not questioning
your logic, but you stated you had collected some numbers, and
I couldn't understand exactly what you talked about.
Presumably you ran some searches and counted some things.
You mention that there are 2x as many macros and half
of the remaining functions should be macros, or something
like that (they "want to have" an indent spec).

I genuinely didn't understand fully.  Can you share whatever
search you used here or in the bug tracker?

I think this is an impacting enough decision you are presenting
here to require some information.

João

[-- Attachment #2: Type: text/html, Size: 1454 bytes --]

Re: Indentation of def*

[Eli Zaretskii]

> From: Stefan Kangas <stefankangas@gmail.com>
> Date: Fri, 15 Oct 2021 03:35:04 -0500
> Cc: luangruo@yahoo.com, larsi@gnus.org, emacs-devel@gnu.org
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > We don't make such announcements in NEWS.  That file describes the
> > changes actually done in the version with which it comes.  If there
> > _is_ a change we make in preparation for some future action, then we
> > can indeed tell that in NEWS.
> 
> I was thinking of something like this line from etc/NEWS.27:
> 
>     ** The ftx font backend driver is now obsolete and will be removed in
>     Emacs 28.
> 
> AFAIK, the above message was just informational and came with no
> functional changes.

No, the underlying actual change was that the ftx font backend is
obsolete.

Re: Indentation of def*

[Lars Ingebrigtsen]

Po Lu <luangruo@yahoo.com> writes:

> That would be a step backwards, at least without a grace period for all
> the code out there to adjust.

Well, the grace period is that this is in Emacs 29 -- so people that
have affected code can start tagging their functions well in advance of
the Emacs 29.1 release as people using the trunk discover functions
that lack

  (declare (indent function))

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Indentation of def*

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

>   (declare (indent function))

(declare (indent defun))

I mean.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Indentation of def*

[Stefan Kangas]

Eli Zaretskii <eliz@gnu.org> writes:

> > AFAIK, the above message was just informational and came with no
> > functional changes.
>
> No, the underlying actual change was that the ftx font backend is
> obsolete.

Again, AFAIK that didn't come with any functional change, it was just
announced in NEWS (see commit f07a4701248).  This arrangement was
suggested in https://debbugs.gnu.org/cgi/bugreport.cgi?bug=34663#23

In any case, I think it could make sense to similarly announce the
obsoletion of this in NEWS.28.  But I don't feel strongly about it or
anything, so feel free to disregard if you don't like the idea.

Re: Indentation of def*

[Stefan Monnier]

Stefan Kangas [2021-10-14 17:50:18] wrote:
> Po Lu <luangruo@yahoo.com> writes:
>> That would be a step backwards, at least without a grace period for all
>> the code out there to adjust.
> I don't think it's a bad idea to announce this change as planned for
> Emacs 29.1 in the Emacs 28.1 NEWS file.

You mean something like:

   ** Deprecate ad-hoc indentation rule for functions/macros named def*
   All macros and functions which expect to be indented similarly to
   `defun` should use something like (declare (indent defun)) instead.

?  We could, but I think the more important change is to make it
concrete with changes in the behavior.

In order to make the change more gradual, here are some ideas:
- Let font-lock put an ugly color on those cases that would be affected
  by the rule change.
- Emit a warning every time we use the ad-hoc def* rule during
  indentation (could even be made fancy, e.g. accompany the warning
  with a button that jumps to the definition and adds the missing
  (declare (indent defun))).
- Make the change conditional on a variable, so that users can
  easily recover the previous behavior until the missing `declare`
  is added.  Could be a boolean var, or a var containing a list
  of offenders.


        Stefan

Re: Indentation of def*

[Eli Zaretskii]

> From: Stefan Kangas <stefankangas@gmail.com>
> Date: Fri, 15 Oct 2021 15:07:11 +0200
> Cc: Po Lu <luangruo@yahoo.com>, Lars Ingebrigtsen <larsi@gnus.org>, 
> 	Emacs developers <emacs-devel@gnu.org>
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > > AFAIK, the above message was just informational and came with no
> > > functional changes.
> >
> > No, the underlying actual change was that the ftx font backend is
> > obsolete.
> 
> Again, AFAIK that didn't come with any functional change

Why does it matter?  The important point is that we actually obsoleted
something; the exact expression of the obsolescence is our internal
business.

> In any case, I think it could make sense to similarly announce the
> obsoletion of this in NEWS.28.  But I don't feel strongly about it or
> anything, so feel free to disregard if you don't like the idea.

_If_ we are going to obsolete that (I have no opinion on that), _then_
saying we plan doing something with that in a future release will be
justified.

Re: Indentation of def*

[Stefan Kangas]

Eli Zaretskii <eliz@gnu.org> writes:

> _If_ we are going to obsolete that (I have no opinion on that), _then_
> saying we plan doing something with that in a future release will be
> justified.

We seem to be in violent agreement.

Re: Indentation of def*

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> A longstanding issue is whether to do anything about this heuristic in
> our indentation code:
>
> (defun definitely-regular (a b))
> (defun still-regular (a b))
>
> (progn
>   (definitely-regular a
>     b)
>   (still-regular a
>                  b))
>
> That is, any function or macro that has a name that starts with "def" is
> indented similarly to a `defun' (etc).

I've now removed this heuristic in Emacs 29.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Indentation of def*

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > That is, any function or macro that has a name that starts with "def" is
  > > indented similarly to a `defun' (etc).

  > I've now removed this heuristic in Emacs 29.

Please keep the heuristic.  It is still useful.

If file A uses a deffoober construct defined in file B, and you visit
file A with file B not loaded, the heuristic will make the uses of
deffoober indent properly.

The idea of this was that if you define something whose name starts with `def',
it should be a defining construct.  If people don't follow that convention,
that's a mistake.

It's a minor mistake, but still, if calls to a function which violates
that convention don't get indented right, the solution is to name it
differently.

Giving those macros explicit indentation properties is ok.  With that
done, the heuristic is no longer needed when their definitions are loaded.

But I think we should still use the heuristic for any def... construct
whose definition is not currenly loaded.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Indentation of def*

[Lars Ingebrigtsen]

Richard Stallman <rms@gnu.org> writes:

> Please keep the heuristic.  It is still useful.
>
> If file A uses a deffoober construct defined in file B, and you visit
> file A with file B not loaded, the heuristic will make the uses of
> deffoober indent properly.

Or wrongly -- functions like `default-boundp' are indented wrongly using
that heuristic.

> The idea of this was that if you define something whose name starts
> with `def', it should be a defining construct.  If people don't follow
> that convention, that's a mistake.

They don't follow that convention.

> Giving those macros explicit indentation properties is ok.  With that
> done, the heuristic is no longer needed when their definitions are loaded.
>
> But I think we should still use the heuristic for any def... construct
> whose definition is not currenly loaded.

I don't see what makes these macros special.  You have the same problem
with macros like `with-*' and `when-let' -- these also indent wrongly if
you haven't loaded their definitions.

The moral is: If you edit Emacs Lisp, you should load the file you're
editing first, otherwise it'll indent badly.  We've plastered over this
"problem" for the def* macros only, but there's so many other macros
that also need special indentation that handling def* macros specially
is a disservice to everybody.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Indentation of def*

[João Távora]

On Wed, Oct 20, 2021 at 8:52 AM Lars Ingebrigtsen <larsi@gnus.org> wrote:
>
> Richard Stallman <rms@gnu.org> writes:
> > Please keep the heuristic.  It is still useful.
> > If file A uses a deffoober construct defined in file B, and you visit
> > file A with file B not loaded, the heuristic will make the uses of
> > deffoober indent properly.
> Or wrongly -- functions like `default-boundp' are indented wrongly using
> that heuristic.

Right. And your example is a _function_.  Why don't you _at least_ keep
the heuristic for macros? Isn't that a worthy compromise, a "no-worse"
situation? It would fix your default-boundp example and, if Richard's
example deffoober is a macro (I'd say there's a good chance it is,
statistically speaking) then the calls to that macro would also be
indented correctly.

What am I missing?

> The moral is: If you edit Emacs Lisp, you should load the file you're
> editing first, otherwise it'll indent badly.

That was always the moral yes.  But now, AFAIU, you're tightening
up those morals a bit, by requiring that Richard edit his code to add
indent specs to his macro definitions like deffoober. If I'm mistaken
in this particular example, please correct me.

João

Re: Indentation of def*

[Lars Ingebrigtsen]

João Távora <joaotavora@gmail.com> writes:

> Right. And your example is a _function_.  Why don't you _at least_ keep
> the heuristic for macros? Isn't that a worthy compromise, a "no-worse"
> situation? It would fix your default-boundp example and, if Richard's
> example deffoober is a macro (I'd say there's a good chance it is,
> statistically speaking) then the calls to that macro would also be
> indented correctly.
>
> What am I missing?

We've been over this.  There's both functions and macros that want to be
indented like a defun, and there both functions a macros that don't want
to be, and sometimes they're called something that starts with def* and
sometimes they don't.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Indentation of def*

[João Távora]

On Wed, Oct 20, 2021 at 9:38 AM Lars Ingebrigtsen <larsi@gnus.org> wrote:
>
> João Távora <joaotavora@gmail.com> writes:
>
> > Right. And your example is a _function_.  Why don't you _at least_ keep
> > the heuristic for macros? Isn't that a worthy compromise, a "no-worse"
> > situation? It would fix your default-boundp example and, if Richard's
> > example deffoober is a macro (I'd say there's a good chance it is,
> > statistically speaking) then the calls to that macro would also be
> > indented correctly.
> > What am I missing?
> We've been over this.  There's both functions and macros that want to be
> indented like a defun, and there both functions a macros that don't want
> to be, and sometimes they're called something that starts with def* and
> sometimes they don't.

Right, I know.  I was referring to the example you gave. I've
also asked you for the listings that you've supposedly collected
on those things so we can make a better informed decision.

But, regardless, is it not so that there are various grades of
change here?

a - do nothing
b - keep heuristic but only for macros, delete for everything else.
c - delete heuristic entirely

From your perspective, isn't 'b' better than 'a'? Yes I get that, for you,
'c' > 'b' and that's why you did it.

But I think there are at least some situations where 'b' is preferrable.

So I'd like to have 'b' please. A compromise.  We could add those
debug specs to the def _functions_ like your example. Whoever
these functions are and however many there are. Again, it'd be
great to have the data. Some data.

Thanks,
João

Re: Indentation of def*

[Lars Ingebrigtsen]

João Távora <joaotavora@gmail.com> writes:

> So I'd like to have 'b' please. A compromise.  We could add those
> debug specs to the def _functions_ like your example. Whoever
> these functions are and however many there are. Again, it'd be
> great to have the data. Some data.

If you want the data, it's there in the Emacs tree.  Look at all
functions/macros named def* with/without indent specs, examine what they
are, and count.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Indentation of def*

[João Távora]

On Wed, Oct 20, 2021 at 10:36 AM Lars Ingebrigtsen <larsi@gnus.org> wrote:
> > So I'd like to have 'b' please. A compromise.  We could add those
> > debug specs to the def _functions_ like your example. Whoever
> > these functions are and however many there are. Again, it'd be
> > great to have the data. Some data.
> If you want the data, it's there in the Emacs tree.  Look at all
> functions/macros named def* with/without indent specs, examine what they
> are, and count.

Counting is easy, but some days ago, you wrote

> .. > about half of them really wants to
> .. > have (indent defun).  (And most of the macros, but not all.)

What are these exceptions you're  referring to?  How can I know
what they are?

Also I think there's a fair amount of code in the so-called "wild" that
probably relies on this heuristic.  Of course I think you understand
that removing the heuristic breaks some of it.  That's a price you're
probably aware of. Woudln't you consider breaking _less_ of it?

https://github.com/search?l=Emacs+Lisp&q=%22defmacro+def%22&type=Code

If you kept the heuristic for macros, I think you wounldn't break these^.

And many, if not all, of the problems of the bugs would still be
resolved, I think.

João

Re: Indentation of def*

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > But, regardless, is it not so that there are various grades of
  > change here?

  > a - do nothing
  > b - keep heuristic but only for macros, delete for everything else.
  > c - delete heuristic entirely

The heuristic defines the default way of indenting calls to entities
when they have no specific indentation rule to override that default.
The question is which default is better.

Following the heuristic by default will lead to better indentation in
some cases, and worse in some cases.  An ugly indentation due to
following the heuristic (when it is wrong) is no worse than an ugly
indentation due to not following it (when it is right).  In either
case, we know what to do to make that problem go away.

So unless we find that the heuristic causes more bad than good, we
should keep it.

I expect it gives us a lot of good and only a little bad.  Though I
have no way of actually measuring this.

  > If you want the data, it's there in the Emacs tree.  Look at all
  > functions/macros named def* with/without indent specs, examine what they
  > are, and count.

This would count the number of constructs which have the potential to
be indented well due to the heuristic, or the potential to be indented
badly due to the heuristic.  But that's the wrong way to evaluate the
amount of good and bad done by the heuristic.

Consider for example this function:

  > Or wrongly -- functions like `default-boundp' are indented wrongly using
  > that heuristic.

Indeed, if you break the line right after `(default-boundp', it
indents one extra space.  That indentation is not bad or confusing,
only slightly different.

But I conjecture it will almost never happen anyway -- that people
break the line there so rarely that it hardly qualifies as a problem.

To evaluate how well the heuristic works, we should look for the cases
which are more likely to occur and would indent in a truly annoying or
misleading way, either due to the heuristic or due to its absence.

  > I don't see what makes these macros special.  You have the same problem
  > with macros like `with-*' and `when-let' -- these also indent wrongly if
  > you haven't loaded their definitions.

That's true.  The convention of macros called `with-...' macros
started in the 1990s, I think.

I suggest we look at adding a few more indentation heuristics.
We could start with `with-...'

Normally the first sexp is the one that should be indented specially.
Are there any `with-...' macros that don't follow that general pattern?
Are there any other function/macros named `with-...'?


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Indentation of def*

[Lars Ingebrigtsen]

Richard Stallman <rms@gnu.org> writes:

> But I conjecture it will almost never happen anyway -- that people
> break the line there so rarely that it hardly qualifies as a problem.

It happens all the time.  Consider:

(let ((not-default
       'bar)
      (default
	'foo))
  )

(Note different indentation of the two values.)

> That's true.  The convention of macros called `with-...' macros
> started in the 1990s, I think.
>
> I suggest we look at adding a few more indentation heuristics.
> We could start with `with-...'

The introduction of the `with-*' macros, added three decades ago without
any such heuristic, shows that these heuristics aren't necessary (or
useful).

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Indentation of def*

[Stefan Kangas]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> The introduction of the `with-*' macros, added three decades ago without
> any such heuristic, shows that these heuristics aren't necessary (or
> useful).

And, again, the correct fix for this is to explicitly say what you want
with an indent declaration.  Unlike a heuristic, this will have no false
positives - the user decides.

Re: Indentation of def*

[João Távora]

On Sun, Oct 24, 2021 at 3:27 PM Stefan Kangas <stefankangas@gmail.com> wrote:
>
> Lars Ingebrigtsen <larsi@gnus.org> writes:
>
> > The introduction of the `with-*' macros, added three decades ago without
> > any such heuristic, shows that these heuristics aren't necessary (or
> > useful).
>
> And, again, the correct fix for this is to explicitly say what you want
> with an indent declaration.  Unlike a heuristic, this will have no false
> positives - the user decides.

If there was NO code in the wild that didn't rely on this
heuristic, I could agree to that.  But it is an undeniable fact
that we do, and that code's indentation will be broken.

So, instead of killing the heuristic altogether, why shouldn't
we adjust it so that it has less false positives? In fact I think
if we tweak the heuristic to only apply to macro definitions
it will probably have zero false positives.  Unless I'm mistaken
there are NO macros whose name starts 'def' for which
we DON'T want to indent their uses like a `defun`.

Yes, that action will still bring some false negatives, i.e. the
indentation of SOME code in the wild will still be broken.
But "some" is better than "a lot of" in my book, so I'm completely
the rationale for NOT doing this less intrusive fix is completely
lost on me.

João

Re: Indentation of def*

[Stefan Monnier]

João Távora [2021-10-24 16:19:54] wrote:
> If there was NO code in the wild that didn't rely on this
> heuristic, I could agree to that.  But it is an undeniable fact
> that we do, and that code's indentation will be broken.

We're bikeshedding here.  The change has a very minor temporary impact
and we're just debating different colors to paint the transition.


        Stefan

Re: Indentation of def*

[Lars Ingebrigtsen]

João Távora <joaotavora@gmail.com> writes:

> Unless I'm mistaken there are NO macros whose name starts 'def' for
> which we DON'T want to indent their uses like a `defun`.

As I've said repeatedly, you are mistaken.  There's
`define-thing-chars', for instance.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Indentation of def*

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 782 bytes --]

On Sun, Oct 24, 2021, 17:55 Lars Ingebrigtsen <larsi@gnus.org> wrote:

> João Távora <joaotavora@gmail.com> writes:
>
> > Unless I'm mistaken there are NO macros whose name starts 'def' for
> > which we DON'T want to indent their uses like a `defun`.
>
> As I've said repeatedly, you are mistaken.  There's
> `define-thing-chars', for instance.
>

Ok. Finally an example. It's that a macro? If so, it's the first time you
provide an example. This is why I asked you for data earlier.

Then i'd day say to add a debug spec to define-thing-chars.  Probably you
already did.

Doesn't change my point. Even ifyou're holding on to a bunch more of these
examples (are you?), there's still the fact that you're breaking
indentation of out-of-tree code.

João

>

[-- Attachment #2: Type: text/html, Size: 1540 bytes --]

Re: Indentation of def*

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

    (let ((not-default
           'bar)
          (default
	    'foo))
      )


(default 'foo) is not a call to the function `default'.
It's a list which is not an expression.
The bug here is that the indentation of that list
pays attention to something about `default'.

The same problem occurs with explicit indentation specs:

(let ((defun
          foo)
      (defmacro
          bar)
      )
  )

The fix would be to make indentation understand
the structure of a binding list.
That might be nontrivial work, but it would make a bigger
improvement in the indentation.

  > The introduction of the `with-*' macros, added three decades ago without
  > any such heuristic, shows that these heuristics aren't necessary (or
  > useful).

It does show something.  I think it shows that if we introduce a
category of macros without any special heuristic for them, the absence
won't draw much in the way of complaints.

But it doesn't show that we are better off not introducing the
heuristic, nor that it is wise to delete such a heuristic if we have
been accustomed to using one.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Indentation of def*

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > So, instead of killing the heuristic altogether, why shouldn't
  > we adjust it so that it has less false positives? In fact I think
  > if we tweak the heuristic to only apply to macro definitions
  > it will probably have zero false positives.

Is it possible to override the heuristic with an indentation property
that specifies "treat it like any other function"?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Indentation of def*

[Lars Ingebrigtsen]

Richard Stallman <rms@gnu.org> writes:

> The fix would be to make indentation understand
> the structure of a binding list.
> That might be nontrivial work, but it would make a bigger
> improvement in the indentation.

It would indeed, and if somebody could fix that, it'd be great.
Meanwhile, the removal of the heuristic makes a swathe of let bindings
indent correctly now.

>   > The introduction of the `with-*' macros, added three decades ago without
>   > any such heuristic, shows that these heuristics aren't necessary (or
>   > useful).
>
> It does show something.  I think it shows that if we introduce a
> category of macros without any special heuristic for them, the absence
> won't draw much in the way of complaints.

This was in response to your claim that the heuristic is necessary
because otherwise the code won't indent correctly before you've loaded
the macro definition.

The `with-*' experience shows that this isn't a problem in practice.
People that edit code will load the code first.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Indentation of def*

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > It would indeed, and if somebody could fix that, it'd be great.
  > Meanwhile, the removal of the heuristic makes a swathe of let bindings
  > indent correctly now.

But other similar let-bindings will indent badly because of inappropriate
application of explicit indentation specs.

Would someone like to try to implement checking the containing list
to see to see if this is really a variable binding in something like `let'?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)