Incorporate package macrostep into Emacs or NonGNU ELPA?

Incorporate package macrostep into Emacs or NonGNU ELPA?

[Jeremy Bryant via Emacs development discussions.]

[-- Attachment #1: Type: text/plain, Size: 966 bytes --]


Please consider the suggestion below

What facility?

"macrostep: interactive macro-expander
macrostep is an Emacs minor mode for interactively stepping through the expansion of macros in Emacs Lisp source code. It lets you see exactly what happens at each step of the expansion process by pretty-printing the expanded forms inline in the source buffer, which is temporarily read-only while macro expansions are visible. "


Where?

Current repo:
This is currently maintained by Jonas in 
https://github.com/emacsorphanage/macrostep

based on a fork of Jon Oddie's repo (original author hasn't responded in
a while) at https://github.com/joddie/macrostep

This is packaged via MELPA.


Who?

Jonas has forked and enhanced the code with contributions from Stefan K
and Stefan M in the last 2y

License?

License is GPLv3


Next steps?
I understand Jonas is supportive of a move to e.g. (eg elpa.git)

Latest package main file attached for convenience in code review

[-- Attachment #2: macrostep.el --]
[-- Type: application/emacs-lisp, Size: 48074 bytes --]

[-- Attachment #3: Type: text/plain, Size: 38 bytes --]


I can volunteer for part of the work

Re: Incorporate package macrostep into Emacs or NonGNU ELPA?

[Stefan Monnier]

> Please consider the suggestion below

http://elpa.nongnu.org/nongnu/macrostep.html

It's been there since 2021 :-)


        Stefan

Re: Incorporate package macrostep into Emacs or NonGNU ELPA?

[Jeremy Bryant]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Please consider the suggestion below
>
> http://elpa.nongnu.org/nongnu/macrostep.html
>
> It's been there since 2021 :-)
>
>
>         Stefan

Thanks Stefan, my fault for not double checking - no excuse!


Which brings us to the other point suggested by Jonas - moving
development & maintenance of macrostep to NonGNU ELPA (on Savannah) itself and
archiving the previous fork.

Would this be of interest?

Re: Incorporate package macrostep into Emacs or NonGNU ELPA?

[Jeremy Bryant]

Jeremy Bryant <jb@jeremybryant.net> writes:

>
> Which brings us to the other point suggested by Jonas - moving
> development & maintenance of macrostep to NonGNU ELPA (on Savannah) itself and
> archiving the previous fork.
>
> Would this be of interest?

Jonathan Oddie has kindly proposed to sign the FSF paperwork once he
receives it.

Given that macrostep is useful for Emacs Lisp macro development, would
there be interest to include in Emacs core?

If so I can volunteer to reach out to other recent contributors, beyond
the original author, for the same purpose?

WDYT?

Re: Incorporate package macrostep into Emacs or NonGNU ELPA?

[Adam Porter]

Hi Jeremy,

> Jonathan Oddie has kindly proposed to sign the FSF paperwork once he
> receives it.
> 
> Given that macrostep is useful for Emacs Lisp macro development, would
> there be interest to include in Emacs core?
> 
> If so I can volunteer to reach out to other recent contributors, beyond
> the original author, for the same purpose?
> 
> WDYT?

To the extent that my input matters, I would certainly be in favor of 
having macrostep in core!

Thanks for working on this.

--Adam

Re: Incorporate package macrostep into Emacs or NonGNU ELPA?

[Stefan Monnier]

Hi Jeremy,

> Which brings us to the other point suggested by Jonas - moving
> development & maintenance of macrostep to NonGNU ELPA (on Savannah)
> itself and archiving the previous fork.

Not sure what you mean by "previous fork".
[ BTW, I only now notice that the "upstream" is called "emacsorphanage".  ]

We can set the upstream to nil and change the headers to say that
development takes place directly in `nongnu.git`, but I'm not sure it's
really an improvement over the status quo.

> Jonathan Oddie has kindly proposed to sign the FSF paperwork once he
> receives it.

Not sure if I understand correctly.  Is he already in the process of
signing the paperwork, or is he waiting for someone to send him the form
for that (in which case, I'd be happy to send it to him)?

> Given that macrostep is useful for Emacs Lisp macro development, would
> there be interest to include in Emacs core?

I don't have a strong opinion either way.

> If so I can volunteer to reach out to other recent contributors, beyond
> the original author, for the same purpose?

That would be awesome.

    % git log elpa/macrostep | grep Author: | sort | uniq -c | sort -n
      1 Author: Chunyang Xu <xuchunyang56@gmail.com>
      1 Author: duianto <duianto@users.noreply.github.com>
      1 Author: John Wiegley <johnw@newartisans.com>
      1 Author: Jonathan Oddie <jonxfield@gmail.com>
      1 Author: Torbjörn Norinder <torbjorn@genunix.se>
      2 Author: Fice T <fice-t@protonmail.com>
      2 Author: Jon Oddie <jonxfield@gmail.com>
      2 Author: Luís Borges de Oliveira <lbo@siscog.pt>
      2 Author: Stefan Monnier <monnier@iro.umontreal.ca>
      3 Author: George Kettleborough <g.kettleborough@member.fsf.org>
      4 Author: Jonathan Oddie <j.j.oddie@gmail.com>
      4 Author: Stefan Kangas <stefankangas@gmail.com>
     12 Author: Luís Oliveira <loliveira@common-lisp.net>
     13 Author: Jonas Bernoulli <jonas@bernoul.li>
     80 Author: joddie <jonxfield@gmail.com>

Of those, Luís Oliveira has signed some paperwork but not for Emacs, and
Fice, Torbjörn, and "duianto" don't appear at all in the
`copyright.list` so we'll need to either ask them to sign the paperwork,
or look at their contributions (to see if they're small enough or have
been replaced since).


        Stefan

Re: Incorporate package macrostep into Emacs or NonGNU ELPA?

[Jeremy Bryant]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Hi Jeremy,
>
>> Which brings us to the other point suggested by Jonas - moving
>> development & maintenance of macrostep to NonGNU ELPA (on Savannah)
>> itself and archiving the previous fork.
>
> Not sure what you mean by "previous fork".
> [ BTW, I only now notice that the "upstream" is called "emacsorphanage".  ]
>
> We can set the upstream to nil and change the headers to say that
> development takes place directly in `nongnu.git`, but I'm not sure it's
> really an improvement over the status quo.

macrostep was originally written by Jonathan Oddie, at
https://github.com/joddie/macrostep
but no longer maintained.  It was forked and improved by Jonas Bernoulli
and others (inc. Stefan K.)
https://github.com/emacsorphanage/macrostep

My original request was to move the 'orphanage' repo to one of the
Savannah repos, in line with a discussion with Jonas:

> I think this is a good idea.  I've considered this myself before, but
> never got around to it.
>
> There are other people that are much more qualified to maintain this
> package, and that certainly includes the core developers.  Stefan Kangas
> (cced) has contributed to the fork before.
>
> I would like to hand of this package to someone else, preferably the
> core team.  If we decide to go down that road, that would mean adding
> a branch to elpa.git and making that the official upstream repository,
> and archiving the repository in the orphanage.
>
>     Cheers,
>     Jonas

>
>> Jonathan Oddie has kindly proposed to sign the FSF paperwork once he
>> receives it.
>
> Not sure if I understand correctly.  Is he already in the process of
> signing the paperwork, or is he waiting for someone to send him the form
> for that (in which case, I'd be happy to send it to him)?

The form has now been sent to Jonathan by Eli.

>> Given that macrostep is useful for Emacs Lisp macro development, would
>> there be interest to include in Emacs core?
>
> I don't have a strong opinion either way.

In another message on this list, Eli is in favour so I will continue
work on this, Emacs core rather than NonGNU.

>
>> If so I can volunteer to reach out to other recent contributors, beyond
>> the original author, for the same purpose?
>
> That would be awesome.
>
>     % git log elpa/macrostep | grep Author: | sort | uniq -c | sort -n
>       1 Author: Chunyang Xu <xuchunyang56@gmail.com>
>       1 Author: duianto <duianto@users.noreply.github.com>
>       1 Author: John Wiegley <johnw@newartisans.com>
>       1 Author: Jonathan Oddie <jonxfield@gmail.com>
>       1 Author: Torbjörn Norinder <torbjorn@genunix.se>
>       2 Author: Fice T <fice-t@protonmail.com>
>       2 Author: Jon Oddie <jonxfield@gmail.com>
>       2 Author: Luís Borges de Oliveira <lbo@siscog.pt>
>       2 Author: Stefan Monnier <monnier@iro.umontreal.ca>
>       3 Author: George Kettleborough <g.kettleborough@member.fsf.org>
>       4 Author: Jonathan Oddie <j.j.oddie@gmail.com>
>       4 Author: Stefan Kangas <stefankangas@gmail.com>
>      12 Author: Luís Oliveira <loliveira@common-lisp.net>
>      13 Author: Jonas Bernoulli <jonas@bernoul.li>
>      80 Author: joddie <jonxfield@gmail.com>
>
> Of those, Luís Oliveira has signed some paperwork but not for Emacs, and
> Fice, Torbjörn, and "duianto" don't appear at all in the
> `copyright.list` so we'll need to either ask them to sign the paperwork,
> or look at their contributions (to see if they're small enough or have
> been replaced since).

I will start work on this.

Re: Incorporate package macrostep into Emacs or NonGNU ELPA?

[Stefan Monnier]

>> That would be awesome.
>>
>>     % git log elpa/macrostep | grep Author: | sort | uniq -c | sort -n
>>       1 Author: Chunyang Xu <xuchunyang56@gmail.com>
>>       1 Author: duianto <duianto@users.noreply.github.com>
>>       1 Author: John Wiegley <johnw@newartisans.com>
>>       1 Author: Jonathan Oddie <jonxfield@gmail.com>
>>       1 Author: Torbjörn Norinder <torbjorn@genunix.se>
>>       2 Author: Fice T <fice-t@protonmail.com>
>>       2 Author: Jon Oddie <jonxfield@gmail.com>
>>       2 Author: Luís Borges de Oliveira <lbo@siscog.pt>
>>       2 Author: Stefan Monnier <monnier@iro.umontreal.ca>
>>       3 Author: George Kettleborough <g.kettleborough@member.fsf.org>
>>       4 Author: Jonathan Oddie <j.j.oddie@gmail.com>
>>       4 Author: Stefan Kangas <stefankangas@gmail.com>
>>      12 Author: Luís Oliveira <loliveira@common-lisp.net>
>>      13 Author: Jonas Bernoulli <jonas@bernoul.li>
>>      80 Author: joddie <jonxfield@gmail.com>
>>
>> Of those, Luís Oliveira has signed some paperwork but not for Emacs, and
>> Fice, Torbjörn, and "duianto" don't appear at all in the
>> `copyright.list` so we'll need to either ask them to sign the paperwork,
>> or look at their contributions (to see if they're small enough or have
>> been replaced since).
>
> I will start work on this.

Thanks.  I have a fair bit of experience looking at those things to try
and whittle down the set of people we need to contact.  Let me know if
I can help.


        Stefan


PS: Also, if/when you send the form to those people, you can now include
an additional entry at the end so *you* get notified when the paperwork
is finally done (by default only the official Emacs maintainers get the
notification).

Re: Incorporate package macrostep into Emacs or NonGNU ELPA?

[Adam Porter]

> PS: Also, if/when you send the form to those people, you can now include
> an additional entry at the end so *you* get notified when the paperwork
> is finally done (by default only the official Emacs maintainers get the
> notification).

Is this documented anywhere?  Or could you show me how to do this?  I've 
been having to wait for contributors to do the copyright assignment for 
some of the packages I maintain on GNU ELPA, and I have to rely on the 
contributor to tell me when their assignment is completed.

Also, am I supposed to be asking the Emacs maintainers to confirm when 
this happens?  Or to confirm when anyone offers a contribution that 
would require CA?  I understand that "the list" is confidential for 
privacy reasons, and for some contributors I can look at emacs.git to 
determine whether they've already done it, but otherwise a package 
maintainer like myself doesn't always know, beyond what the contributor 
says.

Thanks,
Adam

Re: Incorporate package macrostep into Emacs or NonGNU ELPA?

[Eli Zaretskii]

> Date: Sun, 3 Mar 2024 01:26:18 -0600
> Cc: eliz@gnu.org, emacs-devel@gnu.org, j.j.oddie@gmail.com,
>  jb@jeremybryant.net, jonas@bernoul.li, stefan@marxist.se,
>  stefankangas@gmail.com
> From: Adam Porter <adam@alphapapa.net>
> 
> Also, am I supposed to be asking the Emacs maintainers to confirm when 
> this happens?

Yes, please.

Re: Incorporate package macrostep into Emacs or NonGNU ELPA?

[Adam Porter]

On 3/3/24 01:51, Eli Zaretskii wrote:
>> Date: Sun, 3 Mar 2024 01:26:18 -0600
>> Cc: eliz@gnu.org, emacs-devel@gnu.org, j.j.oddie@gmail.com,
>>   jb@jeremybryant.net, jonas@bernoul.li, stefan@marxist.se,
>>   stefankangas@gmail.com
>> From: Adam Porter <adam@alphapapa.net>
>>
>> Also, am I supposed to be asking the Emacs maintainers to confirm when
>> this happens?
> 
> Yes, please.

Ok, should I confirm here, or email certain people privately?

Re: Incorporate package macrostep into Emacs or NonGNU ELPA?

[Eli Zaretskii]

> Date: Sun, 3 Mar 2024 01:53:53 -0600
> Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org, j.j.oddie@gmail.com,
>  jb@jeremybryant.net, jonas@bernoul.li, stefan@marxist.se,
>  stefankangas@gmail.com
> From: Adam Porter <adam@alphapapa.net>
> 
> On 3/3/24 01:51, Eli Zaretskii wrote:
> >> Date: Sun, 3 Mar 2024 01:26:18 -0600
> >> Cc: eliz@gnu.org, emacs-devel@gnu.org, j.j.oddie@gmail.com,
> >>   jb@jeremybryant.net, jonas@bernoul.li, stefan@marxist.se,
> >>   stefankangas@gmail.com
> >> From: Adam Porter <adam@alphapapa.net>
> >>
> >> Also, am I supposed to be asking the Emacs maintainers to confirm when
> >> this happens?
> > 
> > Yes, please.
> 
> Ok, should I confirm here, or email certain people privately?

It doesn't matter.  Your call.  If you email privately, use the
addresses of the Emacs maintainers.

Re: Incorporate package macrostep into Emacs or NonGNU ELPA?

[Stefan Monnier]

>> PS: Also, if/when you send the form to those people, you can now include
>> an additional entry at the end so *you* get notified when the paperwork
>> is finally done (by default only the official Emacs maintainers get the
>> notification).
> Is this documented anywhere?

It's fairly new, and things don't move fast on that side.

> Or could you show me how to do this?

See the last element in the example below.

> I've been having to wait for contributors to do the copyright
> assignment for some of the packages I maintain on GNU ELPA, and I have
> to rely on the contributor to tell me when their assignment
> is completed.

I know, which is why I have negotiated this new element 🙂

> Also, am I supposed to be asking the Emacs maintainers to confirm when this
> happens?

If by "this" you mean the contributor telling you that it's completed,
then yes.  But with the new thingy you should receive that confirmation
directly from the copyright clerk at the same time as the
contributor does, in which case you don't need to.

> I understand that "the list" is confidential for privacy reasons,

Yup, I still haven't managed to negotiate a solution for that 🙁


        Stefan


Please email the following information to assign@gnu.org, and we will send you
the assignment form for your past and future changes.

Please use your full legal name (in ASCII characters) as the subject line of
the message.
----------------------------------------------------------------------
REQUEST: SEND FORM FOR PAST AND FUTURE CHANGES

[What is the name of the program or package you're contributing to?]
Emacs

[Did you copy any files or text written by someone else in these changes?
 Even if that material is free software, we need to know about it.]


[Do you have an employer who might have a basis to claim to own your changes?
 Do you attend a school which might make such a claim?]


[For the copyright registration, what country are you a citizen of?]


[What year were you born?]


[Please write your email address here.]


[Please write your postal address here.]





[Which files have you changed so far, and which new files have you written
so far?]


[Additional people we should notify about the progress of the assignment.]
Stefan Monnier <monnier@gnu.org>

Re: Incorporate package macrostep into Emacs or NonGNU ELPA?

[Ihor Radchenko]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>> PS: Also, if/when you send the form to those people, you can now include
>>> an additional entry at the end so *you* get notified when the paperwork
>>> is finally done (by default only the official Emacs maintainers get the
>>> notification).
>> Is this documented anywhere?
>
> It's fairly new, and things don't move fast on that side.
>
>> Or could you show me how to do this?
>
> See the last element in the example below.
> ...
> [Additional people we should notify about the progress of the assignment.]
> Stefan Monnier <monnier@gnu.org>

Is there any reason why this variant of the form is not linked from
CONTRIBUTE file in Emacs repository? The link there points to
https://git.savannah.gnu.org/cgit/gnulib.git/plain/doc/Copyright/request-assign.future
that does not have the added passage.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Incorporate package macrostep into Emacs or NonGNU ELPA?

[Stefan Monnier]

> Is there any reason why this variant of the form is not linked from
> CONTRIBUTE file in Emacs repository? The link there points to
> https://git.savannah.gnu.org/cgit/gnulib.git/plain/doc/Copyright/request-assign.future
> that does not have the added passage.

As the famous wise man said:

    It's fairly new, and things don't move fast on that side.

IOW, please report it to the gnulib guys.


        Stefan

Re: Incorporate package macrostep into Emacs or NonGNU ELPA?

[Jeremy Bryant]

>> If so I can volunteer to reach out to other recent contributors, beyond
>> the original author, for the same purpose?
>>> That would be awesome.
>>>
>>>     % git log elpa/macrostep | grep Author: | sort | uniq -c | sort -n
>>>       1 Author: Chunyang Xu <xuchunyang56@gmail.com>
>>>       1 Author: duianto <duianto@users.noreply.github.com>
>>>       1 Author: John Wiegley <johnw@newartisans.com>
>>>       1 Author: Jonathan Oddie <jonxfield@gmail.com>
>>>       1 Author: Torbjörn Norinder <torbjorn@genunix.se>
>>>       2 Author: Fice T <fice-t@protonmail.com>
>>>       2 Author: Jon Oddie <jonxfield@gmail.com>
>>>       2 Author: Luís Borges de Oliveira <lbo@siscog.pt>
>>>       2 Author: Stefan Monnier <monnier@iro.umontreal.ca>
>>>       3 Author: George Kettleborough <g.kettleborough@member.fsf.org>
>>>       4 Author: Jonathan Oddie <j.j.oddie@gmail.com>
>>>       4 Author: Stefan Kangas <stefankangas@gmail.com>
>>>      12 Author: Luís Oliveira <loliveira@common-lisp.net>
>>>      13 Author: Jonas Bernoulli <jonas@bernoul.li>
>>>      80 Author: joddie <jonxfield@gmail.com>
>>>
>>> Of those, Luís Oliveira has signed some paperwork but not for Emacs, and
>>> Fice, Torbjörn, and "duianto" don't appear at all in the
>>> `copyright.list` so we'll need to either ask them to sign the paperwork,
>>> or look at their contributions (to see if they're small enough or have
>>> been replaced since).
>>
>> I will start work on this.
>
> Thanks.  I have a fair bit of experience looking at those things to try
> and whittle down the set of people we need to contact.  Let me know if
> I can help.
>
>
>         Stefan

OK, here are some questions on the interpretation of the above.

>       1 Author: duianto <duianto@users.noreply.github.com>
1-line change 7y ago. 2016.  A typo fix.
No apparent usable email address
Can we assume no need to chase, equivalent to:
\"Copyright-paperwork-exempt: yes\"

>       2 Author: Fice T <fice-t@protonmail.com>
1 change 7y ago, 1 line
1 change 7y ago, -5 lines +7lines
Small changes, can we assume no need to chase?

>       1 Author: Torbjörn Norinder <torbjorn@genunix.se>
more than 15 lines, 7m ago, apparently non-trivial patch related to macroexpand-1
To contact, I will do this

>       2 Author: Luís Borges de Oliveira <lbo@siscog.pt>
>      12 Author: Luís Oliveira <loliveira@common-lisp.net>
To contact, I will do this

Re: Incorporate package macrostep into Emacs or NonGNU ELPA?

[Eli Zaretskii]

> From: Jeremy Bryant <jb@jeremybryant.net>
> Cc: "emacs-devel@gnu.org" <emacs-devel@gnu.org>,  j.j.oddie@gmail.com,
>   Stefan Kangas <stefankangas@gmail.com>,  Jonas Bernoulli
>  <jonas@bernoul.li>,  Eli Zaretskii <eliz@gnu.org>
> Date: Sun, 03 Mar 2024 22:40:16 +0000
> 
> >       1 Author: duianto <duianto@users.noreply.github.com>
> 1-line change 7y ago. 2016.  A typo fix.
> No apparent usable email address
> Can we assume no need to chase, equivalent to:
> \"Copyright-paperwork-exempt: yes\"

Yes.

> >       2 Author: Fice T <fice-t@protonmail.com>
> 1 change 7y ago, 1 line
> 1 change 7y ago, -5 lines +7lines
> Small changes, can we assume no need to chase?

Yes.

Re: Incorporate package macrostep into Emacs or NonGNU ELPA?

[Jeremy Bryant]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Jeremy Bryant <jb@jeremybryant.net>
>> Cc: "emacs-devel@gnu.org" <emacs-devel@gnu.org>,  j.j.oddie@gmail.com,
>>   Stefan Kangas <stefankangas@gmail.com>,  Jonas Bernoulli
>>  <jonas@bernoul.li>,  Eli Zaretskii <eliz@gnu.org>
>> Date: Sun, 03 Mar 2024 22:40:16 +0000
>> 
>> >       1 Author: duianto <duianto@users.noreply.github.com>
>> 1-line change 7y ago. 2016.  A typo fix.
>> No apparent usable email address
>> Can we assume no need to chase, equivalent to:
>> \"Copyright-paperwork-exempt: yes\"
>
> Yes.
>
>> >       2 Author: Fice T <fice-t@protonmail.com>
>> 1 change 7y ago, 1 line
>> 1 change 7y ago, -5 lines +7lines
>> Small changes, can we assume no need to chase?
>
> Yes.

As an update, having contacted them, the previously missing 3 material contributors
have kindly now confirmed they sent the request for the FSF paperwork.

Jonathan Oddie - original author.
Torbjörn Norinder <torbjorn@genunix.se>
Luís Oliveira luismbo@gmail.com

Re: Incorporate package macrostep into Emacs or NonGNU ELPA?

[Eli Zaretskii]

> From: Jeremy Bryant <jb@jeremybryant.net>
> Cc: "emacs-devel@gnu.org" <emacs-devel@gnu.org>,  j.j.oddie@gmail.com,
>  Stefan Kangas <stefan@marxist.se>,  Stefan Kangas
>  <stefankangas@gmail.com>,  Jonas Bernoulli <jonas@bernoul.li>, Eli
>  Zaretskii <eliz@gnu.org>
> Date: Thu, 29 Feb 2024 20:44:56 +0000
> 
> Given that macrostep is useful for Emacs Lisp macro development, would
> there be interest to include in Emacs core?

Sounds useful, so I'm in favor.

Re: Incorporate package macrostep into Emacs or NonGNU ELPA?

[Jeremy Bryant]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Jeremy Bryant <jb@jeremybryant.net>
>> Cc: "emacs-devel@gnu.org" <emacs-devel@gnu.org>,  j.j.oddie@gmail.com,
>>  Stefan Kangas <stefan@marxist.se>,  Stefan Kangas
>>  <stefankangas@gmail.com>,  Jonas Bernoulli <jonas@bernoul.li>, Eli
>>  Zaretskii <eliz@gnu.org>
>> Date: Thu, 29 Feb 2024 20:44:56 +0000
>> 
>> Given that macrostep is useful for Emacs Lisp macro development, would
>> there be interest to include in Emacs core?
>
> Sounds useful, so I'm in favor.

OK, I will continue to work towards it.

Re: Incorporate package macrostep into Emacs core

[Jeremy Bryant via Emacs development discussions.]

[-- Attachment #1: Type: text/plain, Size: 719 bytes --]


>>> Given that macrostep is useful for Emacs Lisp macro development, would
>>> there be interest to include in Emacs core?
>>
>> Sounds useful, so I'm in favor.
>
> OK, I will continue to work towards it.

Eli, Stefan,

As I wait for the FSF paperwork to be completed for several
contributors:

Manual?
Should the documentation for macrostep be included in the Emacs Lisp
manual section Macros?  Or a more suitable location?  I volunteer to
write the manual sections.

Code?
The main file is attached for convenience, from the orphanage upstream
(https://github.com/emacsorphanage/macrostep). 
Are any changes needed before this is merged into Emacs?
I volunteer to write some code towards this, please let me know.



[-- Attachment #2: macrostep.el --]
[-- Type: application/emacs-lisp, Size: 48295 bytes --]

Re: Incorporate package macrostep into Emacs core

[Philip Kaludercic]

Jeremy Bryant via "Emacs development discussions." <emacs-devel@gnu.org>
writes:

>>>> Given that macrostep is useful for Emacs Lisp macro development, would
>>>> there be interest to include in Emacs core?
>>>
>>> Sounds useful, so I'm in favor.
>>
>> OK, I will continue to work towards it.
>
> Eli, Stefan,
>
> As I wait for the FSF paperwork to be completed for several
> contributors:
>
> Manual?
> Should the documentation for macrostep be included in the Emacs Lisp
> manual section Macros?  Or a more suitable location?  I volunteer to
> write the manual sections.
>
> Code?
> The main file is attached for convenience, from the orphanage upstream
> (https://github.com/emacsorphanage/macrostep). 
> Are any changes needed before this is merged into Emacs?
> I volunteer to write some code towards this, please let me know.

I have a few comments:

>
> ;;; macrostep.el --- Interactive macro expander  -*- lexical-binding: t; -*-
>
> ;; Copyright (C) 2012-2015 Jon Oddie
> ;; Copyright (C) 2020-2023 Free Software Foundation, Inc.

I guess this should be updated until 2024.

> ;; Author: Jon Oddie <j.j.oddie@gmail.com>
> ;; Url: https://github.com/emacsorphanage/macrostep
> ;; Keywords: lisp, languages, macro, debugging
>
> ;; Package-Version: 0.9.2
> ;; Package-Requires: ((cl-lib "0.5"))
>
> ;; SPDX-License-Identifier: GPL-3.0-or-later
>
> ;; This file is free software: you can redistribute it and/or modify
> ;; it under the terms of the GNU General Public License as published
> ;; by the Free Software Foundation, either version 3 of the License,
> ;; or (at your option) any later version.
> ;;
> ;; This file is distributed in the hope that it will be useful,
> ;; but WITHOUT ANY WARRANTY; without even the implied warranty of
> ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> ;; GNU General Public License for more details.
> ;;
> ;; You should have received a copy of the GNU General Public License
> ;; along with this file.  If not, see <https://www.gnu.org/licenses/>.
>
> ;;; Commentary:
>
> ;; `macrostep' is an Emacs minor mode for interactively stepping through
> ;; the expansion of macros in Emacs Lisp source code.  It lets you see
> ;; exactly what happens at each step of the expansion process by
> ;; pretty-printing the expanded forms inline in the source buffer, which is
> ;; temporarily read-only while macro expansions are visible.  You can
> ;; expand and collapse macro forms one step at a time, and evaluate or
> ;; instrument the expansions for debugging with Edebug as normal (but see
> ;; "Bugs and known limitations", below).  Single-stepping through the
> ;; expansion is particularly useful for debugging macros that expand into
> ;; another macro form.  These can be difficult to debug with Emacs'
> ;; built-in `macroexpand', which continues expansion until the top-level
> ;; form is no longer a macro call.
>
> ;; Both globally-visible macros as defined by `defmacro' and local macros
> ;; bound by `(cl-)macrolet' or another macro-defining form can be expanded.
> ;; Within macro expansions, calls to macros and compiler macros are
> ;; fontified specially: macro forms using `macrostep-macro-face', and
> ;; functions with compiler macros using `macrostep-compiler-macro-face'.
> ;; Uninterned symbols (gensyms) are fontified based on which step in the
> ;; expansion created them, to distinguish them both from normal symbols and
> ;; from other gensyms with the same print name.
>
> ;; As of version 0.9, it is also possible to extend `macrostep' to work
> ;; with other languages with macro systems in addition to Emacs Lisp.  An
> ;; extension for Common Lisp (via SLIME) is in the works; contributions for
> ;; other languages are welcome.  See "Extending macrostep" below for
> ;; details.
>
>
> ;; 1 Key-bindings and usage
> ;; ========================
>
> ;;   The standard keybindings in `macrostep-mode' are the following:
>
> ;;   e, =, RET : expand the macro form following point one step
> ;;   c, u, DEL : collapse the form following point
> ;;   q, C-c C-c: collapse all expanded forms and exit macrostep-mode
> ;;   n, TAB    : jump to the next macro form in the expansion
> ;;   p, M-TAB  : jump to the previous macro form in the expansion
>
> ;;   It's not very useful to enable and disable macrostep-mode directly.
> ;;   Instead, bind `macrostep-expand' to a key in `emacs-lisp-mode-map',
> ;;   for example C-c e:
>
> ;;   ,----
> ;;   | (define-key emacs-lisp-mode-map (kbd "C-c e") 'macrostep-expand)
> ;;   `----
>
> ;;   You can then enter macrostep-mode and expand a macro form completely
> ;;   by typing `C-c e e e ...' as many times as necessary.
>
> ;;   Exit macrostep-mode by typing `q' or `C-c C-c', or by successively
> ;;   typing `c' to collapse all surrounding expansions.
>
>
> ;; 2 Customization options
> ;; =======================
>
> ;;   Type `M-x customize-group RET macrostep RET' to customize options and
> ;;   faces.
>
> ;;   To display macro expansions in a separate window, instead of inline in
> ;;   the source buffer, customize `macrostep-expand-in-separate-buffer' to
> ;;   `t'.  The default is `nil'.  Whichever default behavior is selected,
> ;;   the alternative behavior can be obtained temporarily by giving a
> ;;   prefix argument to `macrostep-expand'.
>
> ;;   To have `macrostep' ignore compiler macros, customize
> ;;   `macrostep-expand-compiler-macros' to `nil'.  The default is `t'.
>
> ;;   Customize the faces `macrostep-macro-face',
> ;;   `macrostep-compiler-macro-face', and `macrostep-gensym-1' through
> ;;   `macrostep-gensym-5' to alter the appearance of macro expansions.
>
>
> ;; 3 Locally-bound macros
> ;; ======================
>
> ;;   As of version 0.9, `macrostep' can expand calls to a locally-bound
> ;;   macro, whether defined by a surrounding `(cl-)macrolet' form, or by
> ;;   another macro-defining macro.  In other words, it is possible to
> ;;   expand the inner `local-macro' forms in both the following examples,
> ;;   whether `local-macro' is defined by an enclosing `cl-macrolet' --
>
> ;;   ,----
> ;;   | (cl-macrolet ((local-macro (&rest args)
> ;;   |                 `(expansion of ,args)))
> ;;   |   (local-macro (do-something)))
> ;;   `----
>
> ;;   -- or by a macro which expands into `cl-macrolet', provided that its
> ;;   definition of macro is evaluated prior to calling `macrostep-expand':
>
> ;;   ,----
> ;;   | (defmacro with-local-macro (&rest body)
> ;;   |   `(cl-macrolet ((local-macro (&rest args)
> ;;   |                    `(expansion of ,args)))
> ;;   |      ,@body))
> ;;   |
> ;;   | (with-local-macro
> ;;   |     (local-macro (do something (else)))
> ;;   `----
>
> ;;   See the `with-js' macro in Emacs's `js.el' for a real example of the
> ;;   latter kind of macro.
>
> ;;   Expansion of locally-bound macros is implemented by instrumenting
> ;;   Emacs Lisp's macro-expander to capture the environment at point.  A
> ;;   similar trick is used to detect macro- and compiler-macro calls within
> ;;   expanded text so that they can be fontified accurately.
>
>
> ;; 4 Expanding sub-forms
> ;; =====================
>
> ;;   By moving point around in the macro expansion using
> ;;   `macrostep-next-macro' and `macrostep-prev-macro' (bound to the `n'
> ;;   and `p' keys), it is possible to expand other macro calls within the
> ;;   expansion before expanding the outermost form.  This can sometimes be
> ;;   useful, although it does not correspond to the real order of macro
> ;;   expansion in Emacs Lisp, which proceeds by fully expanding the outer
> ;;   form to a non-macro form before expanding sub-forms.
>
> ;;   The main reason to expand sub-forms out of order is to help with
> ;;   debugging macros which programmatically expand their arguments in
> ;;   order to rewrite them.  Expanding the arguments of such a macro lets
> ;;   you visualise what the macro definition would compute via
> ;;   `macroexpand-all'.
>
>
> ;; 5 Extending macrostep for other languages
> ;; =========================================
>
> ;;   Since version 0.9, it is possible to extend macrostep to work with
> ;;   other languages besides Emacs Lisp.  In typical Emacs fashion, this is
> ;;   implemented by setting buffer-local variables to different function
> ;;   values.  Six buffer-local variables define the language-specific part
> ;;   of the implementation:
>
> ;;   - `macrostep-sexp-bounds-function'
> ;;   - `macrostep-sexp-at-point-function'
> ;;   - `macrostep-environment-at-point-function'
> ;;   - `macrostep-expand-1-function'
> ;;   - `macrostep-print-function'
> ;;   - `macrostep-macro-form-p-function'
>
> ;;   Typically, an implementation for another language would set these
> ;;   variables in a major-mode hook.  See the docstrings of each variable
> ;;   for details on how each one is called and what it should return.  At a
> ;;   minimum, another language implementation needs to provide
> ;;   `macrostep-sexp-at-point-function', `macrostep-expand-1-function', and
> ;;   `macrostep-print-function'.  Lisp-like languages may be able to reuse
> ;;   the default `macrostep-sexp-bounds-function' if they provide another
> ;;   implementation of `macrostep-macro-form-p-function'.  Languages which
> ;;   do not implement locally-defined macros can set
> ;;   `macrostep-environment-at-point-function' to `ignore'.
>
> ;;   Note that the core `macrostep' machinery only interprets the return
> ;;   value of `macrostep-sexp-bounds-function', so implementations for
> ;;   other languages can use any internal representations of code and
> ;;   environments which is convenient.  Although the terminology is
> ;;   Lisp-specific, there is no reason that implementations could not be
> ;;   provided for non-Lisp languages with macro systems, provided there is
> ;;   some way of identifying macro calls and calling the compiler /
> ;;   preprocessor to obtain their expansions.
>
>
> ;; 6 Bugs and known limitations
> ;; ============================
>
> ;;   You can evaluate and edebug macro-expanded forms and step through the
> ;;   macro-expanded version, but the form that `eval-defun' and friends
> ;;   read from the buffer won't have the uninterned symbols of the real
> ;;   macro expansion.  This will probably work OK with CL-style gensyms,
> ;;   but may cause problems with `make-symbol' symbols if they have the
> ;;   same print name as another symbol in the expansion.  It's possible that
> ;;   using `print-circle' and `print-gensym' could get around this.
>
> ;;   Please send other bug reports and feature requests to the author.
>
>
> ;; 7 Acknowledgements
> ;; ==================
>
> ;;   Thanks to:
> ;;   - John Wiegley for fixing a bug with the face definitions under Emacs
> ;;     24 & for plugging macrostep in his [EmacsConf presentation]!
> ;;   - George Kettleborough for bug reports, and patches to highlight the
> ;;     expanded region and properly handle backquotes.
> ;;   - Nic Ferrier for suggesting support for local definitions within
> ;;     macrolet forms
> ;;   - Luís Oliveira for suggesting and implementing SLIME support
>
> ;;   `macrostep' was originally inspired by J. V. Toups's 'Deep Emacs Lisp'
> ;;   articles ([part 1], [part 2], [screencast]).
>
> ;;   [EmacsConf presentation] http://youtu.be/RvPFZL6NJNQ
>
> ;;   [part 1]
> ;;   http://dorophone.blogspot.co.uk/2011/04/deep-emacs-part-1.html
>
> ;;   [part 2]
> ;;   http://dorophone.blogspot.co.uk/2011/04/deep-emacs-lisp-part-2.html
>
> ;;   [screencast]
> ;;   http://dorophone.blogspot.co.uk/2011/05/monadic-parser-combinators-in-elisp.html
>
>
> ;; 8 Changelog
> ;; ===========

It would be better to convert this into a "News" section so that ELPA
can pick out the changelog.

> ;;   - v0.9.2, 2023-05-12:
> ;;     - name the keymap macrostep-mode-map, fixing a regression in v0.9.1
> ;;   - v0.9.1, 2023-03-12:
> ;;     - bug fixes, cleanup and modernization
> ;;   - v0.9, 2015-10-01:
> ;;     - separate into Elisp-specific and generic components
> ;;     - highlight and expand compiler macros
> ;;     - improve local macro expansion and macro form identification by
> ;;       instrumenting `macroexpand(-all)'
> ;;   - v0.8, 2014-05-29: fix a bug with printing the first element of lists
> ;;   - v0.7, 2014-05-11: expand locally-defined macros within
> ;;     `(cl-)macrolet' forms
> ;;   - v0.6, 2013-05-04: better handling of quote and backquote
> ;;   - v0.5, 2013-04-16: highlight region, maintain cleaner buffer state
> ;;   - v0.4, 2013-04-07: only enter macrostep-mode on successful
> ;;     macro-expansion
> ;;   - v0.3, 2012-10-30: print dotted lists correctly.  autoload
> ;;     definitions.
>
> ;;; Code:
>
> (require 'pp)
> (require 'ring)
> (require 'cl-lib)
>
> \f
> ;;; Constants and dynamically bound variables
> (defvar macrostep-overlays nil
>   "List of all macro stepper overlays in the current buffer.")
> (make-variable-buffer-local 'macrostep-overlays)

Here (and below) you can use defvar-local

> (defvar macrostep-gensym-depth nil
>   "Number of macro expansion levels that have introduced gensyms so far.")
> (make-variable-buffer-local 'macrostep-gensym-depth)
>
> (defvar macrostep-gensyms-this-level nil
>   "Non-nil if gensyms have been encountered during current level of macro expansion.")
> (make-variable-buffer-local 'macrostep-gensyms-this-level)
>
> (defvar macrostep-saved-undo-list nil
>   "Saved value of `buffer-undo-list' upon entering macrostep mode.")
> (make-variable-buffer-local 'macrostep-saved-undo-list)
>
> (defvar macrostep-saved-read-only nil
>   "Saved value of `buffer-read-only' upon entering macrostep mode.")
> (make-variable-buffer-local 'macrostep-saved-read-only)
>
> (defvar macrostep-expansion-buffer nil
>   "Non-nil if the current buffer is a macro-expansion buffer.")
> (make-variable-buffer-local 'macrostep-expansion-buffer)
>
> (defvar macrostep-outer-environment nil
>   "Outermost macro-expansion environment to use in macro-expansion buffers.
>
> This variable is used to save information about any enclosing
> `cl-macrolet' context when a macro form is expanded in a separate
> buffer.")
> (make-variable-buffer-local 'macrostep-outer-environment)
>
> ;;; Customization options and faces
> (defgroup macrostep nil
>   "Interactive macro stepper for Emacs Lisp."
>   :group 'lisp
>   :link '(emacs-commentary-link :tag "commentary" "macrostep.el")
>   :link '(emacs-library-link :tag "lisp file" "macrostep.el")
>   :link '(url-link :tag "web page" "https://github.com/joddie/macrostep"))

This URL seems out-of-date.

>
> (defface macrostep-gensym-1
>   '((((min-colors 16581375)) :foreground "#8080c0" :box t :bold t)
>     (((min-colors 8)) :background "cyan")
>     (t :inverse-video t))
>   "Face for gensyms created in the first level of macro expansion.")
>
> (defface macrostep-gensym-2
>   '((((min-colors 16581375)) :foreground "#8fbc8f" :box t :bold t)
>     (((min-colors 8)) :background "#00cd00")
>     (t :inverse-video t))
>   "Face for gensyms created in the second level of macro expansion.")
>
> (defface macrostep-gensym-3
>   '((((min-colors 16581375)) :foreground "#daa520" :box t :bold t)
>     (((min-colors 8)) :background "yellow")
>     (t :inverse-video t))
>   "Face for gensyms created in the third level of macro expansion.")
>
> (defface macrostep-gensym-4
>   '((((min-colors 16581375)) :foreground "#cd5c5c" :box t :bold t)
>     (((min-colors 8)) :background "red")
>     (t :inverse-video t))
>   "Face for gensyms created in the fourth level of macro expansion.")
>
> (defface macrostep-gensym-5
>   '((((min-colors 16581375)) :foreground "#da70d6" :box t :bold t)
>     (((min-colors 8)) :background "magenta")
>     (t :inverse-video t))
>   "Face for gensyms created in the fifth level of macro expansion.")
>
> (defface macrostep-expansion-highlight-face
>   `((((min-colors 16581375) (background light))
>      ,@(and (>= emacs-major-version 27) '(:extend t))
>      :background "#eee8d5")
>     (((min-colors 16581375) (background dark))
>      ,@(and (>= emacs-major-version 27) '(:extend t))

Is there any harm in adding :extend before Emacs 27?  Also, we won't
need the check in the core.

>      :background "#222222"))
>   "Face for macro-expansion highlight.")
>
> (defface macrostep-macro-face
>   '((t :underline t))
>   "Face for macros in macro-expanded code.")
>
> (defface macrostep-compiler-macro-face
>   '((t :slant italic))
>   "Face for compiler macros in macro-expanded code.")
>
> (defcustom macrostep-expand-in-separate-buffer nil
>   "When non-nil, show expansions in a separate buffer instead of inline."
>   :type 'boolean)
>
> (defcustom macrostep-expand-compiler-macros t
>   "When non-nil, also expand compiler macros."
>   :type 'boolean)
>
> ;; Need the following for making the ring of faces
> (defun macrostep-make-ring (&rest items)
>   "Make a ring containing all of ITEMS with no empty slots."
>   (let ((ring (make-ring (length items))))
>     (mapc (lambda (item) (ring-insert ring item)) (reverse items))
>     ring))

Isn't this `ring-convert-sequence-to-ring'?

>
> (defvar macrostep-gensym-faces
>   (macrostep-make-ring
>    'macrostep-gensym-1 'macrostep-gensym-2 'macrostep-gensym-3
>    'macrostep-gensym-4 'macrostep-gensym-5)
>   "Ring of all macrostepper faces for fontifying gensyms.")
>
> ;; Other modes can enable macrostep by redefining these functions to
> ;; language-specific versions.
> (defvar macrostep-sexp-bounds-function
>   #'macrostep-sexp-bounds
>   "Function to return the bounds of the macro form nearest point.
>
> It will be called with no arguments and should return a cons of
> buffer positions, (START . END).  It should use `save-excursion'
> to avoid changing the position of point.
>
> The default value, `macrostep-sexp-bounds', implements this for
> Emacs Lisp, and may be suitable for other Lisp-like languages.")
> (make-variable-buffer-local 'macrostep-sexp-bounds-function)
>
> (defvar macrostep-sexp-at-point-function
>   #'macrostep-sexp-at-point
>   "Function to return the macro form at point for expansion.
>
> It will be called with two arguments, the values of START and END
> returned by `macrostep-sexp-bounds-function', and with point
> positioned at START.  It should return a value suitable for
> passing as the first argument to `macrostep-expand-1-function'.
>
> The default value, `macrostep-sexp-at-point', implements this for
> Emacs Lisp, and may be suitable for other Lisp-like languages.")
> (make-variable-buffer-local 'macrostep-sexp-at-point-function)
>
> (defvar macrostep-environment-at-point-function
>   #'macrostep-environment-at-point
>   "Function to return the local macro-expansion environment at point.
>
> It will be called with no arguments, and should return a value
> suitable for passing as the second argument to
> `macrostep-expand-1-function'.
>
> The default value, `macrostep-environment-at-point', is specific
> to Emacs Lisp.  For languages which do not implement local
> macro-expansion environments, this should be set to `ignore'
> or `(lambda () nil)'.")
> (make-variable-buffer-local 'macrostep-environment-at-point-function)
>
> (defvar macrostep-expand-1-function
>   #'macrostep-expand-1
>   "Function to perform one step of macro-expansion.
>
> It will be called with two arguments, FORM and ENVIRONMENT, the
> return values of `macrostep-sexp-at-point-function' and
> `macrostep-environment-at-point-function' respectively.  It
> should return the result of expanding FORM by one step as a value
> which is suitable for passing as the argument to
> `macrostep-print-function'.
>
> The default value, `macrostep-expand-1', is specific to Emacs Lisp.")
> (make-variable-buffer-local 'macrostep-expand-1-function)
>
> (defvar macrostep-print-function
>   #'macrostep-pp
>   "Function to pretty-print macro expansions.
>
> It will be called with two arguments, FORM and ENVIRONMENT, the
> return values of `macrostep-sexp-at-point-function' and
> `macrostep-environment-at-point-function' respectively.  It
> should insert a pretty-printed representation at point in the
> current buffer, leaving point just after the inserted
> representation, without altering any other text in the current
> buffer.
>
> The default value, `macrostep-pp', is specific to Emacs Lisp.")
> (make-variable-buffer-local 'macrostep-print-function)
>
> (defvar macrostep-macro-form-p-function
>   #'macrostep-macro-form-p
>   "Function to check whether a form is a macro call.
>
> It will be called with two arguments, FORM and ENVIRONMENT -- the
> return values of `macrostep-sexp-at-point-function' and
> `macrostep-environment-at-point-function' respectively -- and
> should return non-nil if FORM would undergo macro-expansion in
> ENVIRONMENT.
>
> This is called only from `macrostep-sexp-bounds', so it need not
> be provided if a different value is used for
> `macrostep-sexp-bounds-function'.
>
> The default value, `macrostep-macro-form-p', is specific to Emacs Lisp.")
> (make-variable-buffer-local 'macrostep-macro-form-p-function)
>
> \f
> ;;; Define keymap and minor mode
> (define-obsolete-variable-alias 'macrostep-mode-keymap 'macrostep-mode-map "2023")
> (define-obsolete-variable-alias 'macrostep-keymap 'macrostep-mode-map "2022")
> (defvar macrostep-mode-map
>   (let ((map (make-sparse-keymap)))
>     (define-key map (kbd "RET") #'macrostep-expand)
>     (define-key map "=" #'macrostep-expand)
>     (define-key map "e" #'macrostep-expand)
>
>     (define-key map (kbd "DEL") #'macrostep-collapse)
>     (define-key map "u" #'macrostep-collapse)
>     (define-key map "c" #'macrostep-collapse)
>
>     (define-key map (kbd "TAB") #'macrostep-next-macro)
>     (define-key map "n" #'macrostep-next-macro)
>     (define-key map (kbd "M-TAB") #'macrostep-prev-macro)
>     (define-key map "p" #'macrostep-prev-macro)
>
>     (define-key map "q" #'macrostep-collapse-all)
>     (define-key map (kbd "C-c C-c") #'macrostep-collapse-all)
>     map)
>   "Keymap for `macrostep-mode'.")

This could be converted to defvar-keymap.

> ;;;###autoload
> (define-minor-mode macrostep-mode
>   "Minor mode for inline expansion of macros in Emacs Lisp source buffers.
>
> \\<macrostep-mode-map>Progressively expand macro forms with \
> \\[macrostep-expand], collapse them with \\[macrostep-collapse],
> and move back and forth with \\[macrostep-next-macro] and \
> \\[macrostep-prev-macro].  Use \\[macrostep-collapse-all] or collapse all
> visible expansions to quit and return to normal editing.
>
> \\{macrostep-mode-map}"
>   :lighter " Macro-Stepper"
>   :group 'macrostep
>   (if macrostep-mode
>       (progn
>         ;; Disable recording of undo information
>         (setq macrostep-saved-undo-list buffer-undo-list
>               buffer-undo-list t)
>         ;; Remember whether buffer was read-only
>         (setq macrostep-saved-read-only buffer-read-only
>               buffer-read-only t)
>         ;; Set up post-command hook to bail out on leaving read-only
>         (add-hook 'post-command-hook #'macrostep-command-hook nil t)
>         (message (substitute-command-keys "\
> \\<macrostep-mode-map>Entering macro stepper mode. \
> Use \\[macrostep-expand] to expand, \\[macrostep-collapse] to collapse, \
> \\[macrostep-collapse-all] to exit.")))
>
>     ;; Exiting mode
>     (if macrostep-expansion-buffer
>         ;; Kill dedicated expansion buffers
>         (quit-window t)
>       ;; Collapse any remaining overlays
>       (when macrostep-overlays (macrostep-collapse-all))
>       ;; Restore undo info & read-only state
>       (setq buffer-undo-list macrostep-saved-undo-list
>             buffer-read-only macrostep-saved-read-only
>             macrostep-saved-undo-list nil)
>       ;; Remove our post-command hook
>       (remove-hook 'post-command-hook #'macrostep-command-hook t))))
>
> (defun macrostep-command-hook ()
>   "Hook function for use by `post-command hook'.
> Bail out of `macrostep-mode' if the user types
> `\\[read-only-mode]' to make the buffer writable again."
>   (if (not buffer-read-only)
>       (macrostep-mode 0)))
>
> \f
> ;;; Interactive functions
> ;;;###autoload
> (defun macrostep-expand (&optional toggle-separate-buffer)
>   "Expand the macro form following point by one step.
>
> Enters `macrostep-mode' if it is not already active, making the
> buffer temporarily read-only.  If `macrostep-mode' is active and
> the form following point is not a macro form, search forward in
> the buffer and expand the next macro form found, if any.
>
> If optional argument TOGGLE-SEPARATE-BUFFER is non-nil (or set
>  with a prefix argument), the expansion is displayed in a
>  separate buffer instead of inline in the current buffer.
>  Setting `macrostep-expand-in-separate-buffer' to non-nil swaps
>  these two behaviors."
>   (interactive "P")
>   (cl-destructuring-bind (start . end)
>       (funcall macrostep-sexp-bounds-function)
>     (goto-char start)
>     (let* ((sexp (funcall macrostep-sexp-at-point-function start end))
>            (end (copy-marker end))
>            (text (buffer-substring start end))
>            (env (funcall macrostep-environment-at-point-function))
>            (expansion (funcall macrostep-expand-1-function sexp env)))
>
>       ;; Create a dedicated macro-expansion buffer and copy the text to
>       ;; be expanded into it, if required
>       (let ((separate-buffer-p
>              (if toggle-separate-buffer
>                  (not macrostep-expand-in-separate-buffer)
>                macrostep-expand-in-separate-buffer)))
>         (when (and separate-buffer-p (not macrostep-expansion-buffer))
>           (let ((mode major-mode)
>                 (buffer
>                  (get-buffer-create (generate-new-buffer-name "*macro expansion*"))))
>             (set-buffer buffer)

Shouldn't this be a `with-current-buffer'?

>             (funcall mode)
>             (setq macrostep-expansion-buffer t)
>             (setq macrostep-outer-environment env)
>             (save-excursion
>               (setq start (point))
>               (insert text)
>               (setq end (point-marker)))
>             (pop-to-buffer buffer))))
>
>       (unless macrostep-mode (macrostep-mode t))
>       (let ((existing-overlay (macrostep-overlay-at-point))
>             (macrostep-gensym-depth macrostep-gensym-depth)
>             (macrostep-gensyms-this-level nil)
>             priority)
>         (if existing-overlay
>             (progn        ; Expanding part of a previous macro-expansion
>               (setq priority (1+ (overlay-get existing-overlay 'priority)))
>               (setq macrostep-gensym-depth
>                     (overlay-get existing-overlay 'macrostep-gensym-depth)))

Multiple `setq's can be merged into one, so the progn isn't necessary here.

>           ;; Expanding source buffer text
>           (setq priority 1)
>           (setq macrostep-gensym-depth -1))
>
>         (with-silent-modifications
>           (atomic-change-group
>             (let ((inhibit-read-only t))
>               (save-excursion
>                 ;; Insert expansion
>                 (funcall macrostep-print-function expansion env)
>                 ;; Delete the original form
>                 (macrostep-collapse-overlays-in (point) end)
>                 (delete-region (point) end)
>                 ;; Create a new overlay
>                 (let* ((overlay
>                         (make-overlay start
>                                       (if (looking-at "\n")
>                                           (1+ (point))
>                                         (point))))
>                        (highlight-overlay (unless macrostep-expansion-buffer
>                                             (copy-overlay overlay))))
>                   (unless macrostep-expansion-buffer
>                     ;; Highlight the overlay in original source buffers only
>                     (overlay-put highlight-overlay 'face 'macrostep-expansion-highlight-face)
>                     (overlay-put highlight-overlay 'priority -1)
>                     (overlay-put overlay 'macrostep-highlight-overlay highlight-overlay))
>                   (overlay-put overlay 'priority priority)
>                   (overlay-put overlay 'macrostep-original-text text)
>                   (overlay-put overlay 'macrostep-gensym-depth macrostep-gensym-depth)
>                   (push overlay macrostep-overlays))))))))))
>
> (defun macrostep-collapse ()
>   "Collapse the innermost macro expansion near point to its source text.
>
> If no more macro expansions are visible after this, exit
> `macrostep-mode'."
>   (interactive)
>   (let ((overlay (macrostep-overlay-at-point)))
>     (when (not overlay) (error "No macro expansion at point"))
>     (let ((inhibit-read-only t))
>       (with-silent-modifications
>         (atomic-change-group
>           (macrostep-collapse-overlay overlay)))))
>   (if (not macrostep-overlays)

Or `unless'

>       (macrostep-mode 0)))
>
> (defun macrostep-collapse-all ()
>   "Collapse all visible macro expansions and exit `macrostep-mode'."
>   (interactive)
>   (let ((inhibit-read-only t))
>     (with-silent-modifications
>       (dolist (overlay macrostep-overlays)
>         (let ((outermost (= (overlay-get overlay 'priority) 1)))
>           ;; We only need restore the original text for the outermost
>           ;; overlays
>           (macrostep-collapse-overlay overlay (not outermost))))))
>   (setq macrostep-overlays nil)
>   (macrostep-mode 0))
>
> (defun macrostep-next-macro ()
>   "Move point forward to the next macro form in macro-expanded text."
>   (interactive)
>   (let* ((start (if (get-text-property (point) 'macrostep-macro-start)
>                     (1+ (point))
>                   (point)))
>          (next (next-single-property-change start 'macrostep-macro-start)))
>     (if next
>         (goto-char next)
>       (error "No more macro forms found"))))
>
> (defun macrostep-prev-macro ()
>   "Move point back to the previous macro form in macro-expanded text."
>   (interactive)
>   (let (prev)
>     (save-excursion
>       (while
>           (progn
>             (setq prev (previous-single-property-change
>                         (point) 'macrostep-macro-start))
>             (if (or (not prev)
>                     (get-text-property (1- prev) 'macrostep-macro-start))
>                 nil
>               (prog1 t (goto-char prev))))))
>     (if prev
>         (goto-char (1- prev))
>       (error "No previous macro form found"))))
>
> \f
> ;;; Utility functions (not language-specific)
>
> (defun macrostep-overlay-at-point ()
>   "Return the innermost macro stepper overlay at point."
>   (cdr (get-char-property-and-overlay (point) 'macrostep-original-text)))
>
> (defun macrostep-collapse-overlay (overlay &optional no-restore-p)
>   "Collapse macro-expansion buffer OVERLAY and restore the unexpanded source text.
>
> As a minor optimization, does not restore the original source
> text if NO-RESTORE-P is non-nil.  This is safe to do when
> collapsing all the sub-expansions of an outer overlay, since the
> outer overlay will restore the original source itself.
>
> Also removes the overlay from `macrostep-overlays'."
>   (with-current-buffer (overlay-buffer overlay)
>     ;; If we're cleaning up we don't need to bother restoring text
>     ;; or checking for inner overlays to delete
>     (unless no-restore-p
>       (let* ((start (overlay-start overlay))
>              (end (overlay-end overlay))
>              (text (overlay-get overlay 'macrostep-original-text))
>              (sexp-end
>               (copy-marker
>                (if (equal (char-before end) ?\n) (1- end) end))))
>         (macrostep-collapse-overlays-in start end)
>         (goto-char (overlay-start overlay))
>         (save-excursion
>           (insert text)
>           (delete-region (point) sexp-end))))
>     ;; Remove overlay from the list and delete it
>     (setq macrostep-overlays
>           (delq overlay macrostep-overlays))
>     (let ((highlight-overlay (overlay-get overlay 'macrostep-highlight-overlay)))
>       (when highlight-overlay (delete-overlay highlight-overlay)))
>     (delete-overlay overlay)))
>
> (defun macrostep-collapse-overlays-in (start end)
>   "Collapse all macrostepper overlays that are strictly between START and END.
>
> Will not collapse overlays that begin at START and end at END."
>   (dolist (ol (overlays-in start end))
>     (when (and (overlay-buffer ol)        ; collapsing may delete other overlays
>                (> (overlay-start ol) start)
>                (< (overlay-end ol) end)
>                (overlay-get ol 'macrostep-original-text))
>       (macrostep-collapse-overlay ol t))))
>
> \f
> ;;; Emacs Lisp implementation
>
> (defun macrostep-sexp-bounds ()
>   "Find the bounds of the macro form nearest point.
>
> If point is not before an open-paren, moves up to the nearest
> enclosing list.  If the form at point is not a macro call,
> attempts to move forward to the next macro form as determined by
> `macrostep-macro-form-p-function'.
>
> Returns a cons of buffer positions, (START . END)."
>   (save-excursion
>     (if (not (looking-at "[(`]"))
>         (backward-up-list 1))
>     (if (equal (char-before) ?`)
>         (backward-char))
>     (let ((sexp (funcall macrostep-sexp-at-point-function))
>           (env (funcall macrostep-environment-at-point-function)))
>       ;; If this isn't a macro form, try to find the next one in the buffer
>       (unless (funcall macrostep-macro-form-p-function sexp env)
>         (condition-case nil
>             (macrostep-next-macro)
>           (error
>            (if (consp sexp)
>                (error "(%s ...) is not a macro form" (car sexp))
>              (error "Text at point is not a macro form"))))))
>     (cons (point) (scan-sexps (point) 1))))
>
> (defun macrostep-sexp-at-point (&rest _ignore)
>   "Return the sexp near point for purposes of macro-stepper expansion.
>
> If the sexp near point is part of a macro expansion, returns the
> saved text of the macro expansion, and does not read from the
> buffer.  This preserves uninterned symbols in the macro
> expansion, so that they can be fontified consistently.  (See
> `macrostep-print-sexp'.)"
>   (or (get-text-property (point) 'macrostep-expanded-text)
>       (sexp-at-point)))
>
> (defun macrostep-macro-form-p (form environment)
>   "Return non-nil if FORM would be evaluated via macro expansion;
> as considered within ENVIRONMENT.
>
> If FORM is an invocation of a macro defined by `defmacro' or an
> enclosing `cl-macrolet' form, return the symbol `macro'.
>
> If `macrostep-expand-compiler-macros' is non-nil and FORM is a
> call to a function with a compiler macro, return the symbol
> `compiler-macro'.
>
> Otherwise, return nil."
>   (car (macrostep--macro-form-info form environment t)))
>
> (defun macrostep--macro-form-info (form environment &optional inhibit-autoload)
>   "Return information about macro definitions that apply to FORM.
>
> If no macros are involved in the evaluation of FORM within
> ENVIRONMENT, returns nil.  Otherwise, returns a cons (TYPE
> . DEFINITION).
>
> If FORM would be evaluated by a macro defined by `defmacro',
> `cl-macrolet', etc., TYPE is the symbol `macro' and DEFINITION is
> the macro definition, as a function.
>
> If `macrostep-expand-compiler-macros' is non-nil and FORM would
> be compiled using a compiler macro, TYPE is the symbol
> `compiler-macro' and DEFINITION is the function that implements
> the compiler macro.
>
> If FORM is an invocation of an autoloaded macro, the behavior
> depends on the value of INHIBIT-AUTOLOAD.  If INHIBIT-AUTOLOAD is
> nil, the file containing the macro definition will be loaded
> using `load-library' and the macro definition returned as normal.
> If INHIBIT-AUTOLOAD is non-nil, no files will be loaded, and the
> value of DEFINITION in the result will be nil."
>   (if (not (and (consp form)
>                 (symbolp (car form))))
>       `(nil . nil)
>     (let* ((head (car form))
>            (local-definition (assoc-default head environment #'eq)))
>       (if local-definition
>           `(macro . ,local-definition)
>         (let ((compiler-macro-definition
>                (and macrostep-expand-compiler-macros
>                     (or (get head 'compiler-macro)
>                         (get head 'cl-compiler-macro)))))
>           (if (and compiler-macro-definition
>                    (not (eq form
>                             (apply compiler-macro-definition form (cdr form)))))
>               `(compiler-macro . ,compiler-macro-definition)
>             (condition-case nil
>                 (let ((fun (indirect-function head)))
>                   (cl-case (car-safe fun)
>                     ((macro)
>                      `(macro . ,(cdr fun)))
>                     ((autoload)
>                      (when (memq (nth 4 fun) '(macro t))
>                        (if inhibit-autoload
>                            `(macro . nil)
>                          (load-library (nth 1 fun))
>                          (macrostep--macro-form-info form nil))))
>                     (t
>                      `(nil . nil))))
>               (void-function nil))))))))
>
> (defun macrostep-expand-1 (form environment)
>   "Return result of macro-expanding by exactly one step the top level of FORM.
> This is done within ENVIRONMENT.
>
> Unlike `macroexpand', this function does not continue macro
> expansion until a non-macro-call results."
>   (cl-destructuring-bind (type . definition)
>       (macrostep--macro-form-info form environment)
>     (cl-ecase type
>       ((nil)
>        form)
>       ((macro)
>        (apply definition (cdr form)))
>       ((compiler-macro)
>        (let ((expansion (apply definition form (cdr form))))
>          (if (equal form expansion)
>              (error "Form left unchanged by compiler macro")
>            expansion))))))
>
> (put 'macrostep-grab-environment-failed 'error-conditions
>      '(macrostep-grab-environment-failed error))
>
> (defun macrostep-environment-at-point ()
>   "Return the local macro-expansion environment at point, if any.
>
> The local environment includes macros declared by any `macrolet'
> or `cl-macrolet' forms surrounding point, as well as by any macro
> forms which expand into a `macrolet'.
>
> The return value is an alist of elements (NAME . FUNCTION), where
> NAME is the symbol locally bound to the macro and FUNCTION is the
> lambda expression that returns its expansion."
>   ;; If point is on a macro form within an expansion inserted by
>   ;; `macrostep-print-sexp', a local environment may have been
>   ;; previously saved as a text property.
>   (let ((saved-environment
>          (get-text-property (point) 'macrostep-environment)))
>     (if saved-environment
>         saved-environment
>       ;; Otherwise, we (ab)use the macro-expander to return the
>       ;; environment at point.  If point is not at an evaluated
>       ;; position in the containing form,
>       ;; `macrostep-environment-at-point-1' will raise an error, and
>       ;; we back up progressively through the containing forms until
>       ;; it succeeds.
>       (save-excursion
>         (catch 'done
>           (while t
>             (condition-case nil
>                 (throw 'done (macrostep-environment-at-point-1))
>               (macrostep-grab-environment-failed
>                (condition-case nil
>                    (backward-sexp)
>                  (scan-error (backward-up-list)))))))))))
>
> (defun macrostep-environment-at-point-1 ()
>   "Attempt to extract the macro environment that would be active at point.
>
> If point is not at an evaluated position within the containing
> form, raise an error."
>   ;; Macro environments are extracted using Emacs Lisp's builtin
>   ;; macro-expansion machinery.  The form containing point is copied
>   ;; to a temporary buffer, and a call to
>   ;; `--macrostep-grab-environment--' is inserted at point.  This
>   ;; altered form is then fully macro-expanded, in an environment
>   ;; where `--macrostep-grab-environment--' is defined as a macro
>   ;; which throws the environment to a uniquely-generated tag.
>   (let* ((point-at-top-level
>           (save-excursion
>             (while (ignore-errors (backward-up-list) t))
>             (point)))
>          (enclosing-form
>           (buffer-substring point-at-top-level
>                             (scan-sexps point-at-top-level 1)))
>          (position (- (point) point-at-top-level))
>          (tag (make-symbol "macrostep-grab-environment-tag"))
>          (grab-environment '--macrostep-grab-environment--))
>     (if (= position 0)
>         nil
>       (with-temp-buffer
>         (emacs-lisp-mode)
>         (insert enclosing-form)
>         (goto-char (+ (point-min) position))
>         (prin1 `(,grab-environment) (current-buffer))
>         (let ((form (read (copy-marker (point-min)))))
>           (catch tag
>             (cl-letf (((symbol-function #'message) (symbol-function #'format)))

Is this supposed to be an `inhibit-message'?

>               (with-no-warnings
>                 (ignore-errors
>                   (macroexpand-all
>                    `(cl-macrolet ((,grab-environment (&environment env)
>                                     (throw ',tag env)))
>                       ,form)))))
>             (signal 'macrostep-grab-environment-failed nil)))))))
>
> (defun macrostep-collect-macro-forms (form &optional environment)
>   "Identify sub-forms of FORM which undergo macro-expansion.
>
> FORM is an Emacs Lisp form.  ENVIRONMENT is a local environment of
> macro definitions.
>
> The return value is a list of two elements, (MACRO-FORM-ALIST
> COMPILER-MACRO-FORMS).
>
> MACRO-FORM-ALIST is an alist of elements of the form (SUBFORM
> . ENVIRONMENT), where SUBFORM is a form which undergoes
> macro-expansion in the course of expanding FORM, and ENVIRONMENT
> is the local macro environment in force when it is expanded.
>
> COMPILER-MACRO-FORMS is a list of subforms which would be
> compiled using a compiler macro.  Since there is no standard way
> to provide a local compiler-macro definition in Emacs Lisp, no
> corresponding local environments are collected for these.
>
> Forms and environments are extracted from FORM by instrumenting
> Emacs's builtin `macroexpand' function and calling
> `macroexpand-all'."
>   (let* ((macro-form-alist '())
>          (compiler-macro-forms '())
>          (override (lambda (real-macroexpand form environment &rest args)
>                      (let ((expansion
>                             (apply real-macroexpand form environment args)))
>                        (cond ((not (eq expansion form))
>                               (setq macro-form-alist
>                                     (cons (cons form environment)
>                                           macro-form-alist)))
>                              ((and (consp form)
>                                    (symbolp (car form))
>                                    macrostep-expand-compiler-macros
>                                    (not (eq form
>                                             (cl-compiler-macroexpand form))))
>                               (setq compiler-macro-forms
>                                     (cons form compiler-macro-forms))))
>                        expansion))))
>     (cl-macrolet ((with-override (fn &rest body)
>                     `(cl-letf (((symbol-function ,fn)
>                                 (apply-partially override (indirect-function ,fn))))
>                        ,@body))
>                   (with-macroexpand-1 (&rest body)
>                     (if (< emacs-major-version 30)
>                         `(progn ,@body) `(with-override #'macroexpand-1 ,@body)))
>                   (with-macroexpand (&rest body)
>                     `(with-override #'macroexpand ,@body)))
>       (with-macroexpand-1
>        (with-macroexpand
>         (ignore-errors
>           (macroexpand-all form environment)))))
>     (list macro-form-alist compiler-macro-forms)))
>
> (defvar macrostep-collected-macro-form-alist nil
>   "An alist of macro forms and environments.
> Controls the printing of sub-forms in `macrostep-print-sexp'.")
>
> (defvar macrostep-collected-compiler-macro-forms nil
>   "A list of compiler-macro forms to be highlighted in `macrostep-print-sexp'.")
>
> (defun macrostep-pp (sexp environment)
>   "Pretty-print SEXP, fontifying macro forms and uninterned symbols.
> This is done within ENVIRONMENT."
>   (cl-destructuring-bind
>       (macrostep-collected-macro-form-alist
>        macrostep-collected-compiler-macro-forms)
>       (macrostep-collect-macro-forms sexp environment)
>     (let ((print-quoted t))
>       (macrostep-print-sexp sexp)
>       ;; Point is now after the expanded form; pretty-print it
>       (save-restriction
>         (narrow-to-region (scan-sexps (point) -1) (point))
>         (save-excursion
>           (pp-buffer)
>           ;; Remove the extra newline inserted by pp-buffer
>           (goto-char (point-max))
>           (delete-region
>            (point)
>            (save-excursion (skip-chars-backward " \t\n") (point))))
>         ;; Indent the newly-inserted form in context
>         (widen)
>         (save-excursion
>           (backward-sexp)
>           (indent-sexp))))))
>
> ;; This must be defined before `macrostep-print-sexp':
> (defmacro macrostep-propertize (form &rest plist)
>   "Evaluate FORM, applying syntax properties in PLIST to any inserted text."
>   (declare (indent 1)
>            (debug (&rest form)))
>   (let ((start (make-symbol "start")))
>     `(let ((,start (point)))
>        (prog1
>            ,form
>          ,@(cl-loop for (key value) on plist by #'cddr
>                     collect `(put-text-property ,start (point)
>                                                 ,key ,value))))))
>
> (defun macrostep-print-sexp (sexp)
>   "Insert SEXP like `print', fontifying macro forms and uninterned symbols.
>
> Fontifies uninterned symbols and macro forms using
> `font-lock-face' property, and saves the actual text of SEXP's
> sub-forms as the `macrostep-expanded-text' text property so that
> any uninterned symbols can be reused in macro expansions of the
> sub-forms.  See also `macrostep-sexp-at-point'.
>
> Macro and compiler-macro forms within SEXP are identified by
> comparison with the `macrostep-collected-macro-form-alist' and
> `macrostep-collected-compiler-macro-forms' variables, which
> should be dynamically let-bound around calls to this function."
>   (cond
>    ((symbolp sexp)
>     ;; Fontify gensyms
>     (if (not (eq sexp (intern-soft (symbol-name sexp))))
>         (macrostep-propertize
>             (prin1 sexp (current-buffer))
>           'font-lock-face (macrostep-get-gensym-face sexp))
>       ;; Print other symbols as normal
>       (prin1 sexp (current-buffer))))
>
>    ((listp sexp)
>     ;; Print quoted and quasiquoted forms nicely.
>     (let ((head (car sexp)))
>       (cond ((and (eq head 'quote)      ; quote
>                   (= (length sexp) 2))
>              (insert "'")
>              (macrostep-print-sexp (cadr sexp)))
>
>             ((and (eq head '\`)         ; backquote
>                   (= (length sexp) 2))
>              (if (assq sexp macrostep-collected-macro-form-alist)
>                  (macrostep-propertize
>                      (insert "`")
>                    'macrostep-expanded-text sexp
>                    'macrostep-macro-start t
>                    'font-lock-face 'macrostep-macro-face)
>                (insert "`"))
>              (macrostep-print-sexp (cadr sexp)))
>
>             ((and (memq head '(\, \,@)) ; unquote
>                   (= (length sexp) 2))
>              (princ head (current-buffer))
>              (macrostep-print-sexp (cadr sexp)))
>
>             (t                          ; other list form
>              (cl-destructuring-bind (macro? . environment)
>                  (or (assq sexp macrostep-collected-macro-form-alist)
>                      '(nil . nil))
>                (let
>                    ((compiler-macro?
>                      (memq sexp macrostep-collected-compiler-macro-forms)))
>                  (if (or macro? compiler-macro?)
>                      (progn
>                        ;; Save the real expansion as a text property on the
>                        ;; opening paren
>                        (macrostep-propertize
>                         (insert "(")
>                         'macrostep-macro-start t
>                         'macrostep-expanded-text sexp
>                         'macrostep-environment environment)
>                        ;; Fontify the head of the macro
>                        (macrostep-propertize
>                         (macrostep-print-sexp head)
>                         'font-lock-face
>                         (if macro?
>                             'macrostep-macro-face
>                           'macrostep-compiler-macro-face)))
>                    ;; Not a macro form
>                    (insert "(")
>                    (macrostep-print-sexp head))))
>
>              ;; Print remaining list elements
>              (setq sexp (cdr sexp))
>              (when sexp (insert " "))
>              (while sexp
>                (if (listp sexp)
>                    (progn
>                      (macrostep-print-sexp (car sexp))
>                      (when (cdr sexp) (insert " "))
>                      (setq sexp (cdr sexp)))
>                  ;; Print tail of dotted list
>                  (insert ". ")
>                  (macrostep-print-sexp sexp)
>                  (setq sexp nil)))
>              (insert ")")))))
>
>    ;; Print everything except symbols and lists as normal
>    (t (prin1 sexp (current-buffer)))))
>
> (defun macrostep-get-gensym-face (symbol)
>   "Return the face to use in fontifying SYMBOL in printed macro expansions.
>
> All symbols introduced in the same level of macro expansion are
> fontified using the same face (modulo the number of faces; see
> `macrostep-gensym-faces')."
>   (or (get symbol 'macrostep-gensym-face)
>       (progn
>         (if (not macrostep-gensyms-this-level)
>             (setq macrostep-gensym-depth (1+ macrostep-gensym-depth)
>                   macrostep-gensyms-this-level t))
>         (let ((face (ring-ref macrostep-gensym-faces macrostep-gensym-depth)))
>           (put symbol 'macrostep-gensym-face face)
>           face))))
>
> \f
> (provide 'macrostep)
> ;; Local Variables:
> ;; indent-tabs-mode: nil
> ;; End:
> ;;; macrostep.el ends here

Isn't there also a C-preprecossor implementation for macrostep?  Would
the plan be to include that as well?


-- 
	Philip Kaludercic on peregrine

Re: Incorporate package macrostep into Emacs core

[Jeremy Bryant]

[-- Attachment #1: Type: text/plain, Size: 312 bytes --]

Philip Kaludercic <philipk@posteo.net> writes:

Philip,
Thanks for all the comments on the macrostep.el code, I'll work on what
I can.

> Isn't there also a C-preprecossor implementation for macrostep?  Would
> the plan be to include that as well?

Presumably, also.  Any comments on macrostep-c.el (attached)?


[-- Attachment #2: macrostep-c.el --]
[-- Type: application/emacs-lisp, Size: 6627 bytes --]

Re: Incorporate package macrostep into Emacs core

[Philip Kaludercic]

Jeremy Bryant <jb@jeremybryant.net> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
> Philip,
> Thanks for all the comments on the macrostep.el code, I'll work on what
> I can.
>
>> Isn't there also a C-preprecossor implementation for macrostep?  Would
>> the plan be to include that as well?
>
> Presumably, also.  Any comments on macrostep-c.el (attached)?
>
> ;;; macrostep-c.el --- macrostep interface to C preprocessor  -*- lexical-binding: t; -*-
>
> ;; Copyright (C) 2015 Jon Oddie

If included, then this should be updated.

> ;; Author: Jon Oddie <j.j.oddie@gmail.com>
> ;; Url: https://github.com/emacsorphanage/macrostep

And this removed.

> ;; Keywords: c, languages, macro, debugging
>
> ;; SPDX-License-Identifier: GPL-3.0-or-later
>
> ;; This file is free software: you can redistribute it and/or modify
> ;; it under the terms of the GNU General Public License as published
> ;; by the Free Software Foundation, either version 3 of the License,
> ;; or (at your option) any later version.
> ;;
> ;; This file is distributed in the hope that it will be useful,
> ;; but WITHOUT ANY WARRANTY; without even the implied warranty of
> ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> ;; GNU General Public License for more details.
> ;;
> ;; You should have received a copy of the GNU General Public License
> ;; along with this file.  If not, see <https://www.gnu.org/licenses/>.
>
> ;;; Commentary:
>
> ;; A thin wrapper around Emacs's built-in `cmacexp' library to provide
> ;; basic support for expanding C macros using the `macrostep' user
> ;; interface.  To use, position point on a macro use in a C buffer and
> ;; type `M-x macrostep-expand'.  The variables `c-macro-preprocessor'
> ;; and especially `c-macro-cppflags' may need to be set correctly for
> ;; accurate expansion.
>
> ;; This is fairly basic compared to the Emacs Lisp `macrostep'.  In
> ;; particular, there is no step-by-step expansion, since C macros are
> ;; expanded in a single "cpp" pass, and no pretty-printing.
>
> ;; To hide the buffer containing "cpp" warnings (not recommended), you
> ;; could do something like:
> ;;
> ;; (push `(,(regexp-quote macrostep-c-warning-buffer)
> ;;          (display-buffer-no-window))
> ;;       display-buffer-alist)
>
> ;;; Code:
>
> (require 'macrostep)
> (require 'cmacexp)
> (require 'cl-lib)
>
> (require 'subr-x nil t)
> (defalias 'macrostep-c-string-trim
>   (if (fboundp 'string-trim)
>       #'string-trim
>     (lambda (string)
>       (when (string-match "\\`[ \t\n\r]+" string)
> 	(setq string (replace-match "" t t string)))
>       (when (string-match "[ \t\n\r]+\\'" string)
> 	(setq string (replace-match "" t t string)))
>       string)))

We can drop this (or use Compat to provide the function if the package
is distributeted on ELPA).

> (put 'macrostep-c-non-macro 'error-conditions
>      '(macrostep-c-non-macro error))
> (put 'macrostep-c-non-macro 'error-message
>      "Text around point is not a macro call.")
>
> (put 'macrostep-c-expansion-failed 'error-conditions
>      '(macrostep-c-expansion-failed error))
> (put 'macrostep-c-expansion-failed 'error-message
>      "Macro-expansion failed.")
>
> (defvar macrostep-c-warning-buffer "*Macroexpansion Warnings*")
>
> ;;;###autoload
> (defun macrostep-c-mode-hook ()
>   (setq macrostep-sexp-bounds-function
>         #'macrostep-c-sexp-bounds)
>   (setq macrostep-sexp-at-point-function
>         #'macrostep-c-sexp-at-point)
>   (setq macrostep-environment-at-point-function
>         #'ignore)
>   (setq macrostep-expand-1-function
>         #'macrostep-c-expand-1)
>   (setq macrostep-print-function
>         #'macrostep-c-print-function)
>   (add-hook 'macrostep-mode-off-hook
>             #'macrostep-c-mode-off nil t))
>
> (defun macrostep-c-mode-off (&rest _ignore)
>   (when (derived-mode-p 'c-mode)
>     (let ((warning-window
>            (get-buffer-window macrostep-c-warning-buffer)))
>       (when warning-window
>         (quit-window nil warning-window)))))
>
> ;;;###autoload
> (add-hook 'c-mode-hook #'macrostep-c-mode-hook)

This part is suspicious.  First of all, it looks like one is adding a
hook to a hook, but this would unconditionally modify
c-mode-hook, which I don't think is reliable.  Can we find some other
way to update the variables, depending on the major mode?  E.g. in
macrostep.el one could try to intern (format "macrostep-%s-init"
major-mode) and check if the symbol is fboundp?

> (defun macrostep-c-sexp-bounds ()
>   (save-excursion
>     (cl-loop
>      (let ((region (macrostep-c-sexp-bounds-1)))
>        (cond
>          ((null region)
>           (signal 'macrostep-c-non-macro nil))
>          ((macrostep-c-expandable-p region)
>           (cl-return region))
>          (t
>           (condition-case nil
>               (progn
>                 (backward-up-list)
>                 (skip-syntax-backward "-"))
>             (scan-error
>              (signal 'macrostep-c-non-macro nil)))))))))
>
> (defun macrostep-c-sexp-bounds-1 ()
>   (let ((region (bounds-of-thing-at-point 'symbol)))
>     (when region

One could use when-let* in places like this.

>       (cl-destructuring-bind (symbol-start . symbol-end) region
>         (save-excursion
>           (goto-char symbol-end)
>           (if (looking-at "[[:space:]]*(")
>               (cons symbol-start (scan-sexps symbol-end 1))
>               region))))))

The indentation is off here, right?  Might be worth reindenting
everything once.

>
> (defun macrostep-c-expandable-p (region)
>   (cl-destructuring-bind (start . end) region
>     (condition-case nil
>         (cl-destructuring-bind (expansion _warnings)
>             (macrostep-c-expand-region start end)
>           (and (cl-plusp (length expansion))
>                (not (string= expansion (buffer-substring start end)))))
>       (macrostep-c-expansion-failed nil))))
>
> (defun macrostep-c-sexp-at-point (start end)
>   (cons start end))
>
> (defun macrostep-c-expand-1 (region _ignore)
>   (cl-destructuring-bind (start . end) region
>     (cl-destructuring-bind (expansion warnings)
>         (macrostep-c-expand-region start end)
>       (when (cl-plusp (length warnings))
>         (with-current-buffer
>             (get-buffer-create macrostep-c-warning-buffer)
>           (let ((inhibit-read-only t))
>             (erase-buffer)
>             (insert warnings)
>             (goto-char (point-min)))
>           (special-mode)
>           (display-buffer (current-buffer)
>                           '(display-buffer-pop-up-window
>                             (inhibit-same-window . t)
>                             (allow-no-window . t)))))
>       expansion)))
>
> (defun macrostep-c-expand-region (start end)
>   (let ((expansion
>          (condition-case nil
>              (c-macro-expansion start end
>                                 (concat c-macro-preprocessor " "
>                                         c-macro-cppflags))
>            (search-failed
>             (signal 'macrostep-c-expansion-failed nil)))))
>     (with-temp-buffer
>       (save-excursion
>         (insert expansion))
>       (when (looking-at (regexp-quote "/*"))
>         (search-forward "*/"))
>       (let ((warnings (buffer-substring (point-min) (point)))
>             (expansion (buffer-substring (point) (point-max))))
>         (mapcar #'macrostep-c-string-trim (list expansion warnings))))))
>
> (defun macrostep-c-print-function (expansion &rest _ignore)
>   (with-temp-buffer
>     (insert expansion)
>     (let ((exit-code
>            (shell-command-on-region (point-min) (point-max) "indent" nil t)))

When calling indent, it might be nice to provide an option for optional
flags, in case someone prefers some other indentation scheme.

>       (when (zerop exit-code)
>         (setq expansion (macrostep-c-string-trim (buffer-string))))))
>   (insert expansion))
>
> (provide 'macrostep-c)
>
> ;;; macrostep-c.el ends here
>

-- 
	Philip Kaludercic on peregrine

Re: Incorporate package macrostep into Emacs core

[Gerd Möllmann]

Philip Kaludercic <philipk@posteo.net> writes:

>> (add-hook 'c-mode-hook #'macrostep-c-mode-hook)
>
> This part is suspicious.  First of all, it looks like one is adding a
> hook to a hook, but this would unconditionally modify
> c-mode-hook, which I don't think is reliable.  Can we find some other
> way to update the variables, depending on the major mode?  E.g. in
> macrostep.el one could try to intern (format "macrostep-%s-init"
> major-mode) and check if the symbol is fboundp?

Also, it would be nice to support C++ and Objc, which we have in Emacs
(c-mode-common-hook).

Re: Incorporate package macrostep into Emacs core

[Philip Kaludercic]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
>>> (add-hook 'c-mode-hook #'macrostep-c-mode-hook)
>>
>> This part is suspicious.  First of all, it looks like one is adding a
>> hook to a hook, but this would unconditionally modify
>> c-mode-hook, which I don't think is reliable.  Can we find some other
>> way to update the variables, depending on the major mode?  E.g. in
>> macrostep.el one could try to intern (format "macrostep-%s-init"
>> major-mode) and check if the symbol is fboundp?
>
> Also, it would be nice to support C++ and Objc, which we have in Emacs
> (c-mode-common-hook).

From what I understand, this feature is based on cmacexp, which in turn
just uses cpp.  If something like this feature should be supported for
C++ and Objc, then the functionality should first be added to cmacexp or
something analogous (as it is my understanding that this is currently
not the case).

-- 
	Philip Kaludercic on peregrine

Re: Incorporate package macrostep into Emacs core

[Gerd Möllmann]

Philip Kaludercic <philipk@posteo.net> writes:

> Gerd Möllmann <gerd.moellmann@gmail.com> writes:
>
>> Philip Kaludercic <philipk@posteo.net> writes:
>>
>>>> (add-hook 'c-mode-hook #'macrostep-c-mode-hook)
>>>
>>> This part is suspicious.  First of all, it looks like one is adding a
>>> hook to a hook, but this would unconditionally modify
>>> c-mode-hook, which I don't think is reliable.  Can we find some other
>>> way to update the variables, depending on the major mode?  E.g. in
>>> macrostep.el one could try to intern (format "macrostep-%s-init"
>>> major-mode) and check if the symbol is fboundp?
>>
>> Also, it would be nice to support C++ and Objc, which we have in Emacs
>> (c-mode-common-hook).
>
> From what I understand, this feature is based on cmacexp, which in turn
> just uses cpp.  

What's the problem using cpp?

Re: Incorporate package macrostep into Emacs core

[Philip Kaludercic]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
>> Gerd Möllmann <gerd.moellmann@gmail.com> writes:
>>
>>> Philip Kaludercic <philipk@posteo.net> writes:
>>>
>>>>> (add-hook 'c-mode-hook #'macrostep-c-mode-hook)
>>>>
>>>> This part is suspicious.  First of all, it looks like one is adding a
>>>> hook to a hook, but this would unconditionally modify
>>>> c-mode-hook, which I don't think is reliable.  Can we find some other
>>>> way to update the variables, depending on the major mode?  E.g. in
>>>> macrostep.el one could try to intern (format "macrostep-%s-init"
>>>> major-mode) and check if the symbol is fboundp?
>>>
>>> Also, it would be nice to support C++ and Objc, which we have in Emacs
>>> (c-mode-common-hook).
>>
>> From what I understand, this feature is based on cmacexp, which in turn
>> just uses cpp.  
>
> What's the problem using cpp?

I don't know what Objc does, but in the case of C++ my understanding was
that for example Templates were not expanded by C++.  Of course, if that
is not the intention, then never mind my comment.

-- 
	Philip Kaludercic on peregrine

Re: Incorporate package macrostep into Emacs core

[Gerd Möllmann]

Philip Kaludercic <philipk@posteo.net> writes:

> Gerd Möllmann <gerd.moellmann@gmail.com> writes:
>
>> Philip Kaludercic <philipk@posteo.net> writes:
>>
>>> Gerd Möllmann <gerd.moellmann@gmail.com> writes:
>>>
>>>> Philip Kaludercic <philipk@posteo.net> writes:
>>>>
>>>>>> (add-hook 'c-mode-hook #'macrostep-c-mode-hook)
>>>>>
>>>>> This part is suspicious.  First of all, it looks like one is adding a
>>>>> hook to a hook, but this would unconditionally modify
>>>>> c-mode-hook, which I don't think is reliable.  Can we find some other
>>>>> way to update the variables, depending on the major mode?  E.g. in
>>>>> macrostep.el one could try to intern (format "macrostep-%s-init"
>>>>> major-mode) and check if the symbol is fboundp?
>>>>
>>>> Also, it would be nice to support C++ and Objc, which we have in Emacs
>>>> (c-mode-common-hook).
>>>
>>> From what I understand, this feature is based on cmacexp, which in turn
>>> just uses cpp.  
>>
>> What's the problem using cpp?
>
> I don't know what Objc does, but in the case of C++ my understanding was
> that for example Templates were not expanded by C++.  Of course, if that
> is not the intention, then never mind my comment.

The preprocessor is the same for all three languages. 

If by templates, you mean C++ templates -- these have nothing to do with
the preprocessor. They don't work on the level of text.

Re: Incorporate package macrostep into Emacs core

[Jonathan Oddie]

Hi all,

Sorry I’ve been away from this discussion while traveling.

>> Isn't there also a C-preprecossor implementation for macrostep?  Would
>> the plan be to include that as well?
> 
> Presumably, also.  Any comments on macrostep-c.el (attached)?

I’m not sure how useful the C preprocessor implementation is, as I didn’t do too much work on it. It is more of a proof-of-concept for the ability to extend macrostep to different languages. Of course it’s your choice whether you find it worthwhile to include.

There is a Common Lisp implementation maintained as part of SLIME which is more complete, FYI.

I’ve still to sign and return the FSF paperwork but will do so this week.

Jonathan

Re: Incorporate package macrostep into Emacs core

[Jeremy Bryant via Emacs development discussions.]

[-- Attachment #1: Type: text/plain, Size: 1127 bytes --]

Jonathan Oddie <j.j.oddie@gmail.com> writes:

> Hi all,
>
> Sorry I’ve been away from this discussion while traveling.
>
>>> Isn't there also a C-preprecossor implementation for macrostep?  Would
>>> the plan be to include that as well?
>> 
>> Presumably, also.  Any comments on macrostep-c.el (attached)?
>
> I’m not sure how useful the C preprocessor implementation is, as I
> didn’t do too much work on it. It is more of a proof-of-concept for
> the ability to extend macrostep to different languages. Of course it’s
> your choice whether you find it worthwhile to include.
>
> There is a Common Lisp implementation maintained as part of SLIME
> which is more complete, FYI.

Thank you for clarifying.  On this basis I will only work on adding the
main Lisp related macrostep file for the time being.


> I’ve still to sign and return the FSF paperwork but will do so this week.
>
> Jonathan

Thanks in advance!



What about another file - should the tests file be included in Emacs as part of the main Lisp macrostep?


Should test files be added in the Emacs tree test/lisp/?

[-- Attachment #2: macrostep-test.el --]
[-- Type: application/emacs-lisp, Size: 17072 bytes --]

Re: Incorporate package macrostep into Emacs core

[Jeremy Bryant]

> What about another file - should the tests file be included in Emacs
> as part of the main Lisp macrostep?
>
>
> Should test files be added in the Emacs tree test/lisp/?
>
> [2. application/emacs-lisp; macrostep-test.el]...

Stefan/Philip,

I notice that in NonGNU ELPA's elpa-package there is this 'ignored-files' line:
 (macrostep		:url "https://github.com/emacsorphanage/macrostep"
  :ignored-files ("macrostep-test.el"))


Can you confirm if this means that as part of the move of macrostep to
core, we don't need this test file in core Emacs?

Re: Incorporate package macrostep into Emacs core

[Stefan Monnier]

Jeremy Bryant [2024-03-22 20:47:06] wrote:
> I notice that in NonGNU ELPA's elpa-package there is this 'ignored-files' line:
>  (macrostep		:url "https://github.com/emacsorphanage/macrostep"
>   :ignored-files ("macrostep-test.el"))
>
> Can you confirm if this means that as part of the move of macrostep to
> core, we don't need this test file in core Emacs?

No, I cannot confirm, on the contrary, we do want this file in
`emacs.git` (at the "mirror place" in `test/`, as usual).

The above `:ignored-files` is because someone did not want to force
every user to download the tests as well.


        Stefan

Re: Incorporate package macrostep into Emacs core

[Eli Zaretskii]

> From: Jeremy Bryant <jb@jeremybryant.net>
> Cc: monnier@iro.umontreal.ca,  emacs-devel@gnu.org,  j.j.oddie@gmail.com,
>   stefan@marxist.se,  stefankangas@gmail.com,  jonas@bernoul.li
> Date: Sun, 17 Mar 2024 21:48:08 +0000
> 
> Manual?
> Should the documentation for macrostep be included in the Emacs Lisp
> manual section Macros?

Yes, I think so.

Please also provide a suitable entry for NEWS.

> Code?
> The main file is attached for convenience, from the orphanage upstream
> (https://github.com/emacsorphanage/macrostep). 
> Are any changes needed before this is merged into Emacs?
> I volunteer to write some code towards this, please let me know.

Please add :version tags to all the defcustom's and defface's.

> (define-obsolete-variable-alias 'macrostep-mode-keymap 'macrostep-mode-map "2023")
> (define-obsolete-variable-alias 'macrostep-keymap 'macrostep-mode-map "2022")

The years there should be changed to Emacs versions, I think.

> (defvar macrostep-mode-map
>   (let ((map (make-sparse-keymap)))
>     (define-key map (kbd "RET") #'macrostep-expand)
>     (define-key map "=" #'macrostep-expand)
>     (define-key map "e" #'macrostep-expand)

Bonus points for converting this into defvar-keymap.

> ;; Local Variables:
> ;; indent-tabs-mode: nil
> ;; End:

I think this should be deleted, as this is now the default in ELisp
buffers.

Re: Incorporate package macrostep into Emacs core

[Stefan Monnier]

>> (define-obsolete-variable-alias 'macrostep-mode-keymap 'macrostep-mode-map "2023")
>> (define-obsolete-variable-alias 'macrostep-keymap 'macrostep-mode-map "2022")
> The years there should be changed to Emacs versions, I think.

Hmm... I don't think it would make sense to make it refer to "the
version of Emacs that was current when the change was made" since
macrostep was not included in Emacs back then.


        Stefan

Re: Incorporate package macrostep into Emacs core

[Jeremy Bryant]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Jeremy Bryant <jb@jeremybryant.net>
>> Cc: monnier@iro.umontreal.ca,  emacs-devel@gnu.org,  j.j.oddie@gmail.com,
>>   stefan@marxist.se,  stefankangas@gmail.com,  jonas@bernoul.li
>> Date: Sun, 17 Mar 2024 21:48:08 +0000
>> 
>> Manual?
>> Should the documentation for macrostep be included in the Emacs Lisp
>> manual section Macros?
>
> Yes, I think so.
>
> Please also provide a suitable entry for NEWS.
>
>> Code?
>> The main file is attached for convenience, from the orphanage upstream
>> (https://github.com/emacsorphanage/macrostep). 
>> Are any changes needed before this is merged into Emacs?
>> I volunteer to write some code towards this, please let me know.
>
> Please add :version tags to all the defcustom's and defface's.
>
>> (define-obsolete-variable-alias 'macrostep-mode-keymap
>> 'macrostep-mode-map "2023")
>> (define-obsolete-variable-alias 'macrostep-keymap 'macrostep-mode-map "2022")
>
> The years there should be changed to Emacs versions, I think.
>
>> (defvar macrostep-mode-map
>>   (let ((map (make-sparse-keymap)))
>>     (define-key map (kbd "RET") #'macrostep-expand)
>>     (define-key map "=" #'macrostep-expand)
>>     (define-key map "e" #'macrostep-expand)
>
> Bonus points for converting this into defvar-keymap.
>
>> ;; Local Variables:
>> ;; indent-tabs-mode: nil
>> ;; End:
>
> I think this should be deleted, as this is now the default in ELisp
> buffers.

OK, I'll start working on these.

For the macrostep commits, is it easier for future integration that this
is done externally and submitted in one go, or would something like a
new macrostep branch in the Emacs tree be more appropriate?

Re: Incorporate package macrostep into Emacs core

[Eli Zaretskii]

> From: Jeremy Bryant <jb@jeremybryant.net>
> Cc: monnier@iro.umontreal.ca,  emacs-devel@gnu.org,  j.j.oddie@gmail.com,
>  stefankangas@gmail.com,  jonas@bernoul.li
> Date: Mon, 18 Mar 2024 22:58:02 +0000
> 
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: Jeremy Bryant <jb@jeremybryant.net>
> >> Cc: monnier@iro.umontreal.ca,  emacs-devel@gnu.org,  j.j.oddie@gmail.com,
> >>   stefan@marxist.se,  stefankangas@gmail.com,  jonas@bernoul.li
> >> Date: Sun, 17 Mar 2024 21:48:08 +0000
> >> 
> >> Manual?
> >> Should the documentation for macrostep be included in the Emacs Lisp
> >> manual section Macros?
> >
> > Yes, I think so.
> >
> > Please also provide a suitable entry for NEWS.
> >
> >> Code?
> >> The main file is attached for convenience, from the orphanage upstream
> >> (https://github.com/emacsorphanage/macrostep). 
> >> Are any changes needed before this is merged into Emacs?
> >> I volunteer to write some code towards this, please let me know.
> >
> > Please add :version tags to all the defcustom's and defface's.
> >
> >> (define-obsolete-variable-alias 'macrostep-mode-keymap
> >> 'macrostep-mode-map "2023")
> >> (define-obsolete-variable-alias 'macrostep-keymap 'macrostep-mode-map "2022")
> >
> > The years there should be changed to Emacs versions, I think.
> >
> >> (defvar macrostep-mode-map
> >>   (let ((map (make-sparse-keymap)))
> >>     (define-key map (kbd "RET") #'macrostep-expand)
> >>     (define-key map "=" #'macrostep-expand)
> >>     (define-key map "e" #'macrostep-expand)
> >
> > Bonus points for converting this into defvar-keymap.
> >
> >> ;; Local Variables:
> >> ;; indent-tabs-mode: nil
> >> ;; End:
> >
> > I think this should be deleted, as this is now the default in ELisp
> > buffers.
> 
> OK, I'll start working on these.

Thanks.

> For the macrostep commits, is it easier for future integration that this
> is done externally and submitted in one go, or would something like a
> new macrostep branch in the Emacs tree be more appropriate?

A branch is preferable if you want people to be able to use and test
the package before it lands.  If this package is already in use by
enough people, so you can be reasonably sure it doesn't have any grave
problems, a branch is not required, and you can submit everything as a
single patch.

Re: Incorporate package macrostep into Emacs core

[Jeremy Bryant]

[-- Attachment #1: Type: text/plain, Size: 474 bytes --]


Eli Zaretskii <eliz@gnu.org> writes:

>> From: Jeremy Bryant <jb@jeremybryant.net>
>> Cc: monnier@iro.umontreal.ca,  emacs-devel@gnu.org,  j.j.oddie@gmail.com,
>>   stefan@marxist.se,  stefankangas@gmail.com,  jonas@bernoul.li
>> Date: Sun, 17 Mar 2024 21:48:08 +0000
>> 
>> Manual?
>> Should the documentation for macrostep be included in the Emacs Lisp
>> manual section Macros?
>
> Yes, I think so.
>

Please find attached a patch for the manual.
Any comments welcome.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-new-manual-section-on-macrostep.patch --]
[-- Type: text/x-diff, Size: 4750 bytes --]

From cef9258e824d7a20c8364417c9debd8b2d5133fe Mon Sep 17 00:00:00 2001
From: Jeremy Bryant <jb@jeremybryant.net>
Date: Thu, 18 Apr 2024 21:03:30 +0100
Subject: [PATCH] Add new manual section on macrostep

* doc/lispref/macros.texi (Macros):
Describe macrostep's usage to explore and write macros.

This is based on Jonathan's Oddie's documentation in the macrostep package

Co-authored-by: Jonathan Oddie <j.j.oddie@gmail.com>
---
 doc/lispref/macros.texi | 69 ++++++++++++++++++++++++++++++++++-------
 1 file changed, 58 insertions(+), 11 deletions(-)

diff --git a/doc/lispref/macros.texi b/doc/lispref/macros.texi
index 659dba17524..06e098a8389 100644
--- a/doc/lispref/macros.texi
+++ b/doc/lispref/macros.texi
@@ -23,13 +23,14 @@ Macros
 instead.  @xref{Inline Functions}.
 
 @menu
-* Simple Macro::            A basic example.
-* Expansion::               How, when and why macros are expanded.
-* Compiling Macros::        How macros are expanded by the compiler.
-* Defining Macros::         How to write a macro definition.
-* Problems with Macros::    Don't evaluate the macro arguments too many times.
+* Simple Macro::                A basic example.
+* Expansion::                   How, when and why macros are expanded.
+* Compiling Macros::            How macros are expanded by the compiler.
+* Defining Macros::             How to write a macro definition.
+* Problems with Macros::        Don't evaluate the macro arguments too many times.
                               Don't hide the user's variables.
-* Indenting Macros::        Specifying how to indent macro calls.
+* Indenting Macros::            Specifying how to indent macro calls.
+* macrostep: interactive macro-expander::
 @end menu
 
 @node Simple Macro
@@ -262,12 +263,12 @@ Problems with Macros
 trouble, and rules to follow to avoid trouble.
 
 @menu
-* Wrong Time::             Do the work in the expansion, not in the macro.
-* Argument Evaluation::    The expansion should evaluate each macro arg once.
-* Surprising Local Vars::  Local variable bindings in the expansion
+* Wrong Time::                  Do the work in the expansion, not in the macro.
+* Argument Evaluation::         The expansion should evaluate each macro arg once.
+* Surprising Local Vars::       Local variable bindings in the expansion
                               require special care.
-* Eval During Expansion::  Don't evaluate them; put them in the expansion.
-* Repeated Expansion::     Avoid depending on how many times expansion is done.
+* Eval During Expansion::       Don't evaluate them; put them in the expansion.
+* Repeated Expansion::          Avoid depending on how many times expansion is done.
 @end menu
 
 @node Wrong Time
@@ -640,3 +641,49 @@ Indenting Macros
 number, @kbd{C-M-q} need not recalculate indentation for the following
 lines until the end of the list.
 @end table
+
+
+@node macrostep: interactive macro-expander
+@section macrostep: interactive macro-expander
+
+You can use the package @code{macrostep} to explore macro definitions, and
+help write new macros, using @kbd{M-x macrostep-expand}.
+
+@ifnottex
+@kbd{macrostep} is an Emacs minor mode for interactively stepping
+through the expansion of macros in Emacs Lisp source code.  It lets you
+see exactly what happens at each step of the expansion process by
+pretty-printing the expanded forms inline in the source buffer, which is
+temporarily read-only while macro expansions are visible.  You can
+expand and collapse macro forms one step at a time, and evaluate or
+instrument the expansions for debugging with Edebug as normal.
+Single-stepping through the expansion is particularly useful for
+debugging macros that expand into another macro form.  These can be
+difficult to debug with @code{macroexpand}, which continues expansion
+until the top-level form is no longer a macro call.
+
+The standard keybindings in @code{macrostep-mode} are the following:
+
+@itemize @minus
+@item
+e, =, RET : expand the macro form following point one step
+@item
+c, u, DEL : collapse the form following point
+@item
+q, C-c C-c: collapse all expanded forms and exit macrostep-mode
+@item
+n, TAB    : jump to the next macro form in the expansion
+@item
+p, M-TAB  : jump to the previous macro form in the expansion
+@end itemize
+
+It's not very useful to enable and disable macrostep-mode directly.
+Instead, bind `macrostep-expand' to a key in `emacs-lisp-mode-map',
+for example @kbd{C-c e}.
+
+You can then enter @code{macrostep-mode} and expand a macro form completely
+by typing @kbd{C-c e e e}@dots as many times as necessary.
+
+Exit macrostep-mode by typing @kbd{q} or @kbd{C-c C-c}, or by successively
+typing @kbd{c} to collapse all surrounding expansions.
+@end ifnottex
-- 
2.42.0

Re: Incorporate package macrostep into Emacs core

[Eli Zaretskii]

> From: Jeremy Bryant <jb@jeremybryant.net>
> Cc: monnier@iro.umontreal.ca,  emacs-devel@gnu.org
> Date: Thu, 18 Apr 2024 22:19:36 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: Jeremy Bryant <jb@jeremybryant.net>
> >> Cc: monnier@iro.umontreal.ca,  emacs-devel@gnu.org,  j.j.oddie@gmail.com,
> >>   stefan@marxist.se,  stefankangas@gmail.com,  jonas@bernoul.li
> >> Date: Sun, 17 Mar 2024 21:48:08 +0000
> >> 
> >> Manual?
> >> Should the documentation for macrostep be included in the Emacs Lisp
> >> manual section Macros?
> >
> > Yes, I think so.
> >
> 
> Please find attached a patch for the manual.
> Any comments welcome.

Thanks, see below.

> * doc/lispref/macros.texi (Macros):
> Describe macrostep's usage to explore and write macros.

This is filled sub-optimally; please use change-log-mode to help you
fill better.

> This is based on Jonathan's Oddie's documentation in the macrostep package

Likewise here: this line is too long.

> -* Simple Macro::            A basic example.
> -* Expansion::               How, when and why macros are expanded.
> -* Compiling Macros::        How macros are expanded by the compiler.
> -* Defining Macros::         How to write a macro definition.
> -* Problems with Macros::    Don't evaluate the macro arguments too many times.
> +* Simple Macro::                A basic example.
> +* Expansion::                   How, when and why macros are expanded.
> +* Compiling Macros::            How macros are expanded by the compiler.
> +* Defining Macros::             How to write a macro definition.
> +* Problems with Macros::        Don't evaluate the macro arguments too many times.
>                                Don't hide the user's variables.
> -* Indenting Macros::        Specifying how to indent macro calls.
> +* Indenting Macros::            Specifying how to indent macro calls.
> +* macrostep: interactive macro-expander::

I'd prefer not to change whitespace here.  I see no reason for it in
this case.

Also, any change in the menus requires a corresponding change in
elisp.texi, where we have the @detailmenu.

>  @menu
> -* Wrong Time::             Do the work in the expansion, not in the macro.
> -* Argument Evaluation::    The expansion should evaluate each macro arg once.
> -* Surprising Local Vars::  Local variable bindings in the expansion
> +* Wrong Time::                  Do the work in the expansion, not in the macro.
> +* Argument Evaluation::         The expansion should evaluate each macro arg once.
> +* Surprising Local Vars::       Local variable bindings in the expansion
>                                require special care.
> -* Eval During Expansion::  Don't evaluate them; put them in the expansion.
> -* Repeated Expansion::     Avoid depending on how many times expansion is done.
> +* Eval During Expansion::       Don't evaluate them; put them in the expansion.
> +* Repeated Expansion::          Avoid depending on how many times expansion is done.

Unnecessary whitespace changes again.

> +@node macrostep: interactive macro-expander
> +@section macrostep: interactive macro-expander
> +
> +You can use the package @code{macrostep} to explore macro definitions, and
   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
"You can use the @code{macrostep} package" is better.

Alternatively, say "@code{macrostep-mode}" instead; no need to mention
that it's a package.

> +help write new macros, using @kbd{M-x macrostep-expand}.
> +
> +@ifnottex

This @ifnottex makes this section very small in the printed manual,
which I think is undesirable.  Instead, I'd move the sentence above to
the parent chapter, and have the entire section be inside @ifnottex
(also in menus).

> +@kbd{macrostep} is an Emacs minor mode for interactively stepping

There's no point of having "Emacs" there, since this manual is about
Emacs Lisp.

> +through the expansion of macros in Emacs Lisp source code.  It lets you
> +see exactly what happens at each step of the expansion process by
> +pretty-printing the expanded forms inline in the source buffer, which is
> +temporarily read-only while macro expansions are visible.  You can
> +expand and collapse macro forms one step at a time, and evaluate or
> +instrument the expansions for debugging with Edebug as normal.
                                                       ^^^^^^^^^
"as usual"

> +The standard keybindings in @code{macrostep-mode} are the following:
> +
> +@itemize @minus
> +@item
> +e, =, RET : expand the macro form following point one step

This will produce sub-optimal markup.  I suggest to use "@table @kbd"
instead.

Also, our style is to mention the command bound to the keys in
parentheses, and also index each such command.

> +It's not very useful to enable and disable macrostep-mode directly.
> +Instead, bind `macrostep-expand' to a key in `emacs-lisp-mode-map',
                 ^^^^^^^^^^^^^^^^^^             ^^^^^^^^^^^^^^^^^^^^^
These two should be in @code instead of in quotes `like this'.

> +by typing @kbd{C-c e e e}@dots as many times as necessary.

@dots{} (not "@dots") should be inside @kbd.

And finally, two more questions:

  . should this be in the user manual instead? it sounds like a
    user-level feature, not Lisp programming level feature
  . how is this mode different from "C-x C-k SPC", which is already
    described in the user manual as a similar feature?

Re: Incorporate package macrostep into Emacs core

[Jeremy Bryant]

Eli Zaretskii <eliz@gnu.org> writes:
>> From: Jeremy Bryant <jb@jeremybryant.net>

>> >> Should the documentation for macrostep be included in the Emacs Lisp
>> >> manual section Macros?
>> >
>> > Yes, I think so.

For convenience, to recap from a month ago, this facility is about Lisp
macros, not keyboard macros.  macrostep is useful for Emacs Lisp macro
expansion or exploration, and development.


>> From: Jeremy Bryant <jb@jeremybryant.net>
>> * doc/lispref/macros.texi (Macros):
>> Describe macrostep's usage to explore and write macros.
>
> This is filled sub-optimally; please use change-log-mode to help you
> fill better.

Thank you for the pointer, I will use in future.

For this commit I have used magit-generate-changelog, is this suboptimal?

(..)

Thank you for all the comments on style, I will work on that.

(...)

> And finally, two more questions:
>
>   . should this be in the user manual instead? it sounds like a
>     user-level feature, not Lisp programming level feature

Sure, perhaps this is more suited.

I initially followed your confirmation to write in the Emacs Lisp manual
(top of this message), but indeed this may belong more appropriately in
the Emacs manual.  How about in "(emacs) Programs"?

Please confirm your preference either way and I'll continue the rewrite.

>   . how is this mode different from "C-x C-k SPC", which is already
>     described in the user manual as a similar feature?

Thanks, I'll be clearer in the next iteration. This facility is about
Lisp macros, not keyboard macros.  ('C-x C-k SPC runs the command
kmacro-step-edit-macro').  I'll improve the documentation in the next round.

Re: Incorporate package macrostep into Emacs core

[Stefan Monnier]

> I initially followed your confirmation to write in the Emacs Lisp manual
> (top of this message), but indeed this may belong more appropriately in
> the Emacs manual.  How about in "(emacs) Programs"?
> Please confirm your preference either way and I'll continue the rewrite.

If it works only for Emacs Lisp macros, then it makes sense to put it in
the ELisp manual.  But if it provides a "generic" interface with
backends for various languages, then it might make sense to document it in
the Emacs manual.  And the two are not necessarily mutually exclusive
unless one makes the other redundant.


        Stefan

Re: Incorporate package macrostep into Emacs core

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Eli Zaretskii <eliz@gnu.org>,  emacs-devel@gnu.org
> Date: Fri, 19 Apr 2024 18:26:11 -0400
> 
> > I initially followed your confirmation to write in the Emacs Lisp manual
> > (top of this message), but indeed this may belong more appropriately in
> > the Emacs manual.  How about in "(emacs) Programs"?
> > Please confirm your preference either way and I'll continue the rewrite.
> 
> If it works only for Emacs Lisp macros, then it makes sense to put it in
> the ELisp manual.

It's a minor mode, and the documentation describes commands of that
mode.  I don't think the ELisp manual is a good place for such
features.  We have "Programs" in the user manual, which describes
features for developing programs using Emacs, and I think it's a good
place for describing this feature.

Re: Incorporate package macrostep into Emacs core

[Adam Porter]

>> If it works only for Emacs Lisp macros, then it makes sense to put it in
>> the ELisp manual.
> 
> It's a minor mode, and the documentation describes commands of that
> mode.  I don't think the ELisp manual is a good place for such
> features.  We have "Programs" in the user manual, which describes
> features for developing programs using Emacs, and I think it's a good
> place for describing this feature.

Not that I necessarily disagree, but I think it warrants at least a 
mention in the Elisp manual section about macros, because it's very 
helpful for understanding how macros work.  If it were only mentioned in 
the Emacs manual, new Elispers who are reading up on macros could easily 
overlook this great tool.

Re: Incorporate package macrostep into Emacs core

[Eli Zaretskii]

> From: Jeremy Bryant <jb@jeremybryant.net>
> Cc: monnier@iro.umontreal.ca,  emacs-devel@gnu.org
> Date: Fri, 19 Apr 2024 20:30:30 +0100
> 
> >   . should this be in the user manual instead? it sounds like a
> >     user-level feature, not Lisp programming level feature
> 
> Sure, perhaps this is more suited.
> 
> I initially followed your confirmation to write in the Emacs Lisp manual
> (top of this message), but indeed this may belong more appropriately in
> the Emacs manual.  How about in "(emacs) Programs"?

Yes, a section under "Programs" sounds good.

Thanks.

Re: Incorporate package macrostep into Emacs core

[Jeremy Bryant]

[-- Attachment #1: Type: text/plain, Size: 858 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Jeremy Bryant <jb@jeremybryant.net>
>> Cc: monnier@iro.umontreal.ca,  emacs-devel@gnu.org
>> Date: Fri, 19 Apr 2024 20:30:30 +0100
>> 
>> >   . should this be in the user manual instead? it sounds like a
>> >     user-level feature, not Lisp programming level feature
>> 
>> Sure, perhaps this is more suited.
>> 
>> I initially followed your confirmation to write in the Emacs Lisp manual
>> (top of this message), but indeed this may belong more appropriately in
>> the Emacs manual.  How about in "(emacs) Programs"?
>
> Yes, a section under "Programs" sounds good.
>
> Thanks.

Please find attached a revised patch for consideration.  Any feedback on
style and conventions welcome.

Text is adapted from the macrostep package - I believe Jonathan's FSF
paperwork is still outstanding but is expected soon.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-new-manual-section-on-macrostep.patch --]
[-- Type: text/x-diff, Size: 4750 bytes --]

From cef9258e824d7a20c8364417c9debd8b2d5133fe Mon Sep 17 00:00:00 2001
From: Jeremy Bryant <jb@jeremybryant.net>
Date: Thu, 18 Apr 2024 21:03:30 +0100
Subject: [PATCH] Add new manual section on macrostep

* doc/lispref/macros.texi (Macros):
Describe macrostep's usage to explore and write macros.

This is based on Jonathan's Oddie's documentation in the macrostep package

Co-authored-by: Jonathan Oddie <j.j.oddie@gmail.com>
---
 doc/lispref/macros.texi | 69 ++++++++++++++++++++++++++++++++++-------
 1 file changed, 58 insertions(+), 11 deletions(-)

diff --git a/doc/lispref/macros.texi b/doc/lispref/macros.texi
index 659dba17524..06e098a8389 100644
--- a/doc/lispref/macros.texi
+++ b/doc/lispref/macros.texi
@@ -23,13 +23,14 @@ Macros
 instead.  @xref{Inline Functions}.
 
 @menu
-* Simple Macro::            A basic example.
-* Expansion::               How, when and why macros are expanded.
-* Compiling Macros::        How macros are expanded by the compiler.
-* Defining Macros::         How to write a macro definition.
-* Problems with Macros::    Don't evaluate the macro arguments too many times.
+* Simple Macro::                A basic example.
+* Expansion::                   How, when and why macros are expanded.
+* Compiling Macros::            How macros are expanded by the compiler.
+* Defining Macros::             How to write a macro definition.
+* Problems with Macros::        Don't evaluate the macro arguments too many times.
                               Don't hide the user's variables.
-* Indenting Macros::        Specifying how to indent macro calls.
+* Indenting Macros::            Specifying how to indent macro calls.
+* macrostep: interactive macro-expander::
 @end menu
 
 @node Simple Macro
@@ -262,12 +263,12 @@ Problems with Macros
 trouble, and rules to follow to avoid trouble.
 
 @menu
-* Wrong Time::             Do the work in the expansion, not in the macro.
-* Argument Evaluation::    The expansion should evaluate each macro arg once.
-* Surprising Local Vars::  Local variable bindings in the expansion
+* Wrong Time::                  Do the work in the expansion, not in the macro.
+* Argument Evaluation::         The expansion should evaluate each macro arg once.
+* Surprising Local Vars::       Local variable bindings in the expansion
                               require special care.
-* Eval During Expansion::  Don't evaluate them; put them in the expansion.
-* Repeated Expansion::     Avoid depending on how many times expansion is done.
+* Eval During Expansion::       Don't evaluate them; put them in the expansion.
+* Repeated Expansion::          Avoid depending on how many times expansion is done.
 @end menu
 
 @node Wrong Time
@@ -640,3 +641,49 @@ Indenting Macros
 number, @kbd{C-M-q} need not recalculate indentation for the following
 lines until the end of the list.
 @end table
+
+
+@node macrostep: interactive macro-expander
+@section macrostep: interactive macro-expander
+
+You can use the package @code{macrostep} to explore macro definitions, and
+help write new macros, using @kbd{M-x macrostep-expand}.
+
+@ifnottex
+@kbd{macrostep} is an Emacs minor mode for interactively stepping
+through the expansion of macros in Emacs Lisp source code.  It lets you
+see exactly what happens at each step of the expansion process by
+pretty-printing the expanded forms inline in the source buffer, which is
+temporarily read-only while macro expansions are visible.  You can
+expand and collapse macro forms one step at a time, and evaluate or
+instrument the expansions for debugging with Edebug as normal.
+Single-stepping through the expansion is particularly useful for
+debugging macros that expand into another macro form.  These can be
+difficult to debug with @code{macroexpand}, which continues expansion
+until the top-level form is no longer a macro call.
+
+The standard keybindings in @code{macrostep-mode} are the following:
+
+@itemize @minus
+@item
+e, =, RET : expand the macro form following point one step
+@item
+c, u, DEL : collapse the form following point
+@item
+q, C-c C-c: collapse all expanded forms and exit macrostep-mode
+@item
+n, TAB    : jump to the next macro form in the expansion
+@item
+p, M-TAB  : jump to the previous macro form in the expansion
+@end itemize
+
+It's not very useful to enable and disable macrostep-mode directly.
+Instead, bind `macrostep-expand' to a key in `emacs-lisp-mode-map',
+for example @kbd{C-c e}.
+
+You can then enter @code{macrostep-mode} and expand a macro form completely
+by typing @kbd{C-c e e e}@dots as many times as necessary.
+
+Exit macrostep-mode by typing @kbd{q} or @kbd{C-c C-c}, or by successively
+typing @kbd{c} to collapse all surrounding expansions.
+@end ifnottex
-- 
2.42.0

Re: Incorporate package macrostep into Emacs core

[Eli Zaretskii]

> From: Jeremy Bryant <jb@jeremybryant.net>
> Cc: monnier@iro.umontreal.ca,  emacs-devel@gnu.org
> Date: Tue, 23 Apr 2024 22:37:50 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: Jeremy Bryant <jb@jeremybryant.net>
> >> Cc: monnier@iro.umontreal.ca,  emacs-devel@gnu.org
> >> Date: Fri, 19 Apr 2024 20:30:30 +0100
> >> 
> >> >   . should this be in the user manual instead? it sounds like a
> >> >     user-level feature, not Lisp programming level feature
> >> 
> >> Sure, perhaps this is more suited.
> >> 
> >> I initially followed your confirmation to write in the Emacs Lisp manual
> >> (top of this message), but indeed this may belong more appropriately in
> >> the Emacs manual.  How about in "(emacs) Programs"?
> >
> > Yes, a section under "Programs" sounds good.
> >
> > Thanks.
> 
> Please find attached a revised patch for consideration.  Any feedback on
> style and conventions welcome.
> 
> Text is adapted from the macrostep package - I believe Jonathan's FSF
> paperwork is still outstanding but is expected soon.

Thanks, but it looks like you attached the wrong patch?  It seems like
it's the same patch you sent the first time, not a revised one after
all the comments above, where we agreed to have this in the Emacs user
manual instead.

Re: Incorporate package macrostep into Emacs core

[Jeremy Bryant]

[-- Attachment #1: Type: text/plain, Size: 1417 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Jeremy Bryant <jb@jeremybryant.net>
>> Cc: monnier@iro.umontreal.ca,  emacs-devel@gnu.org
>> Date: Tue, 23 Apr 2024 22:37:50 +0100
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> From: Jeremy Bryant <jb@jeremybryant.net>
>> >> Cc: monnier@iro.umontreal.ca,  emacs-devel@gnu.org
>> >> Date: Fri, 19 Apr 2024 20:30:30 +0100
>> >> 
>> >> >   . should this be in the user manual instead? it sounds like a
>> >> >     user-level feature, not Lisp programming level feature
>> >> 
>> >> Sure, perhaps this is more suited.
>> >> 
>> >> I initially followed your confirmation to write in the Emacs Lisp manual
>> >> (top of this message), but indeed this may belong more appropriately in
>> >> the Emacs manual.  How about in "(emacs) Programs"?
>> >
>> > Yes, a section under "Programs" sounds good.
>> >
>> > Thanks.
>> 
>> Please find attached a revised patch for consideration.  Any feedback on
>> style and conventions welcome.
>> 
>> Text is adapted from the macrostep package - I believe Jonathan's FSF
>> paperwork is still outstanding but is expected soon.
>
> Thanks, but it looks like you attached the wrong patch?  It seems like
> it's the same patch you sent the first time, not a revised one after
> all the comments above, where we agreed to have this in the Emacs user
> manual instead.

Absolutely, sorry, attached is the revised patch I have worked on.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-Macrostep-section-in-manual.patch --]
[-- Type: text/x-diff, Size: 3724 bytes --]

From 0ed5971e7a54abe299386ce53e681b16cdb135c5 Mon Sep 17 00:00:00 2001
From: Jeremy Bryant <jb@jeremybryant.net>
Date: Tue, 23 Apr 2024 22:21:07 +0100
Subject: [PATCH] Add Macrostep section in manual

Macrostep is an Emacs Lisp macro-expander useful for exploring and
writing macros.  This is based on Jonathan's Oddie's documentation
in the macrostep package.

	* doc/emacs/programs.texi (Programs): Add Macrostep section.
	* doc/emacs/emacs.texi (Top): 	Add detailed menu reference.
---
 doc/emacs/emacs.texi    |  3 ++
 doc/emacs/programs.texi | 61 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 64 insertions(+)

diff --git a/doc/emacs/emacs.texi b/doc/emacs/emacs.texi
index 7d77f13ab21..d4d0a753576 100644
--- a/doc/emacs/emacs.texi
+++ b/doc/emacs/emacs.texi
@@ -694,6 +694,9 @@ Top
 @ifnottex
 * Fortran::             Fortran mode and its special features.
 @end ifnottex
+@ifnottex
+* Macrostep::           Interactive Lisp macro-expander.
+@end ifnottex
 
 Top-Level Definitions, or Defuns
 
diff --git a/doc/emacs/programs.texi b/doc/emacs/programs.texi
index de28a9f1dd4..15cfffc289b 100644
--- a/doc/emacs/programs.texi
+++ b/doc/emacs/programs.texi
@@ -45,6 +45,10 @@ Programs
 @ifnottex
 * Fortran::             Fortran mode and its special features.
 @end ifnottex
+@ifnottex
+* Macrostep::           Interactive Lisp macro-expander.
+@end ifnottex
+
 @end menu
 
 @node Program Modes
@@ -2235,3 +2239,60 @@ Asm Mode
 @ifnottex
 @include fortran-xtra.texi
 @end ifnottex
+
+@ifnottex
+@node Macrostep
+@section Macrostep: interactive Lisp macro-expander
+
+You can use @code{macrostep-mode} to explore Lisp macro definitions, and
+help write new macros, using @kbd{M-x macrostep-expand} when point is
+over a macro.
+
+@kbd{macrostep} is a minor mode for interactively stepping through the
+expansion of macros in Emacs Lisp source code.  It lets you see exactly
+what happens at each step of the expansion process by pretty-printing
+the expanded forms inline in the source buffer, which is temporarily
+read-only while macro expansions are visible.  You can expand and
+collapse macro forms one step at a time, and evaluate or instrument the
+expansions for debugging with Edebug as usual.  Single-stepping through
+the expansion is particularly useful for debugging macros that expand
+into another macro form.  These can be difficult to debug with
+@code{macroexpand}, which continues expansion until the top-level form
+is no longer a macro call.
+
+The standard keybindings in @code{macrostep-mode} are the following:
+@table @kbd
+@item e
+@itemx =
+@itemx RET
+@findex macrostep-expand
+Expand the macro form following point one step (@code{macrostep-expand}).
+@item c
+@itemx u
+@itemx DEL
+@findex macrostep-collapse
+Collapse the form following point (@code{macrostep-collapse}).
+@item n
+@itemx TAB
+@findex macrostep-next-macro
+Jump to the next macro form in the expansion (@code{macrostep-next-macro}).
+@item p
+@itemx M-TAB
+@findex macrostep-previous-macro
+Jump to the previous macro form in the expansion (@code{macrostep-previous-macro}).
+@item q
+@itemx C-c C-c
+@findex macrostep-collapse-all
+Collapse all expanded forms and exit macrostep-mode (@code{macrostep-collapse-all}).
+@end table
+
+It's not very useful to enable and disable macrostep-mode directly.
+Instead, bind @code{macrostep-expand} to a key in
+@code{emacs-lisp-mode-map}, for example @kbd{C-c e}.
+
+You can then enter @code{macrostep-mode} and expand a macro form
+completely by typing @kbd{C-c e e e@dots{}} as many times as necessary.
+
+Exit macrostep-mode by typing @kbd{q} or @kbd{C-c C-c}, or by
+successively typing @kbd{c} to collapse all surrounding expansions.
+@end ifnottex
-- 
2.42.0

Re: Incorporate package macrostep into Emacs core

[Eshel Yaron]

Hello Jeremy,

Jeremy Bryant <jb@jeremybryant.net> writes:

> ...attached is the revised patch I have worked on.

A few minor comments:

> From 0ed5971e7a54abe299386ce53e681b16cdb135c5 Mon Sep 17 00:00:00 2001
> From: Jeremy Bryant <jb@jeremybryant.net>
> Date: Tue, 23 Apr 2024 22:21:07 +0100
> Subject: [PATCH] Add Macrostep section in manual
>
> Macrostep is an Emacs Lisp macro-expander useful for exploring and
> writing macros.  This is based on Jonathan's Oddie's documentation
> in the macrostep package.
>
>       * doc/emacs/programs.texi (Programs): Add Macrostep section.
>       * doc/emacs/emacs.texi (Top):   Add detailed menu reference.
> ---
>  doc/emacs/emacs.texi    |  3 ++
>  doc/emacs/programs.texi | 61 +++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 64 insertions(+)
>
> diff --git a/doc/emacs/emacs.texi b/doc/emacs/emacs.texi
> index 7d77f13ab21..d4d0a753576 100644
> --- a/doc/emacs/emacs.texi
> +++ b/doc/emacs/emacs.texi
> @@ -694,6 +694,9 @@ Top
>  @ifnottex
>  * Fortran::             Fortran mode and its special features.
>  @end ifnottex
> +@ifnottex
> +* Macrostep::           Interactive Lisp macro-expander.

IIUC, this is specific to Emacs Lisp, right?  If so, I suggest writing
"Elisp" or "Emacs Lisp" instead of "Lisp" here, for clarity.

> +@end ifnottex
>
>  Top-Level Definitions, or Defuns
>
> diff --git a/doc/emacs/programs.texi b/doc/emacs/programs.texi
> index de28a9f1dd4..15cfffc289b 100644
> --- a/doc/emacs/programs.texi
> +++ b/doc/emacs/programs.texi
> @@ -45,6 +45,10 @@ Programs
>  @ifnottex
>  * Fortran::             Fortran mode and its special features.
>  @end ifnottex
> +@ifnottex
> +* Macrostep::           Interactive Lisp macro-expander.

Likewise here.

> +@end ifnottex
> +
>  @end menu
>
>  @node Program Modes
> @@ -2235,3 +2239,60 @@ Asm Mode
>  @ifnottex
>  @include fortran-xtra.texi
>  @end ifnottex
> +
> +@ifnottex
> +@node Macrostep
> +@section Macrostep: interactive Lisp macro-expander
> +

I'd open with a few more words of motivation, e.g. "Emacs Lisp code that
you read and write often makes use of macros (@pxref{Macros,,,elisp})."

> +You can use @code{macrostep-mode} to explore Lisp macro definitions, and
> +help write new macros...

Since macrostep-mode is not intended to be enabled directly, I suggest
putting less emphasis on the minor mode and more on the feature itself,
so maybe something like "Emacs comes with a feature called
@dfn{Macrostep} that helps you explore and write new macros"

> ...using @kbd{M-x macrostep-expand} when point is
> +over a macro.

Here I think that "when point is over a macro" could be made a little
more precise.  How about "when point is inside a macro form" or
something along these lines?

> +@kbd{macrostep} is a minor mode for interactively stepping through the
> +expansion of macros in Emacs Lisp source code.

"Macrostep lets you step through the expansion of macros interactively."

> It lets you see exactly
> +what happens at each step of the expansion process by pretty-printing
> +the expanded forms inline in the source buffer, which is temporarily
> +read-only while macro expansions are visible.  You can expand and
> +collapse macro forms one step at a time, and evaluate or instrument the
> +expansions for debugging with Edebug as usual.

Maybe add a reference to Edebug?

> Single-stepping through
> +the expansion is particularly useful for debugging macros that expand
> +into another macro form.  These can be difficult to debug with
> +@code{macroexpand}, which continues expansion until the top-level form
> +is no longer a macro call.

The comparison with macroexpand seems a bit like a straw-man argument in
favor of Macrostep, what about macroexpand-1?  IMO, we can just remove
these two sentences.  The utility of Macrostep is clear enough as is.

> +
> +The standard keybindings in @code{macrostep-mode} are the following:
> +@table @kbd
> +@item e
> +@itemx =
> +@itemx RET
> +@findex macrostep-expand
> +Expand the macro form following point one step (@code{macrostep-expand}).

"following point"?  ISTM that this command expands the first macro form
containing point, no?

> +@item c
> +@itemx u
> +@itemx DEL
> +@findex macrostep-collapse
> +Collapse the form following point (@code{macrostep-collapse}).

Likewise.

> +@item n
> +@itemx TAB
> +@findex macrostep-next-macro
> +Jump to the next macro form in the expansion (@code{macrostep-next-macro}).
> +@item p
> +@itemx M-TAB
> +@findex macrostep-previous-macro
> +Jump to the previous macro form in the expansion (@code{macrostep-previous-macro}).
> +@item q
> +@itemx C-c C-c
> +@findex macrostep-collapse-all
> +Collapse all expanded forms and exit macrostep-mode (@code{macrostep-collapse-all}).
> +@end table
> +
> +It's not very useful to enable and disable macrostep-mode directly.

This sentence isn't necessary IMO.

> +Instead, bind @code{macrostep-expand} to a key in
> +@code{emacs-lisp-mode-map}, for example @kbd{C-c e}.

Maybe add example code for how to do that?  Or just refer to some
relevant part of the manual for defining key bindings.
> +
> +You can then enter @code{macrostep-mode} and expand a macro form
> +completely by typing @kbd{C-c e e e@dots{}} as many times as necessary.
> +
> +Exit macrostep-mode by typing @kbd{q} or @kbd{C-c C-c}, or by
> +successively typing @kbd{c} to collapse all surrounding expansions.
> +@end ifnottex


Best regards,

Eshel

Re: Incorporate package macrostep into Emacs core

[Eli Zaretskii]

> From: Jeremy Bryant <jb@jeremybryant.net>
> Cc: monnier@iro.umontreal.ca,  emacs-devel@gnu.org
> Date: Thu, 25 Apr 2024 22:27:56 +0100
> 
> Absolutely, sorry, attached is the revised patch I have worked on.

Thanks, a few minor comments below.

> Subject: [PATCH] Add Macrostep section in manual
> 
> Macrostep is an Emacs Lisp macro-expander useful for exploring and
> writing macros.  This is based on Jonathan's Oddie's documentation
> in the macrostep package.
> 
> 	* doc/emacs/programs.texi (Programs): Add Macrostep section.
> 	* doc/emacs/emacs.texi (Top): 	Add detailed menu reference.
 ^^^^^^^                             ^^^
Those TABs should not be there in the commit log message.

> +You can use @code{macrostep-mode} to explore Lisp macro definitions, and
> +help write new macros, using @kbd{M-x macrostep-expand} when point is
> +over a macro.

We say "when point is in a macro".  "Over" is reserved for mouse
pointer.

> +@kbd{macrostep} is a minor mode for interactively stepping through the
        ^^^^^^^^^
"macrostep-mode", I guess?  And why @kbd? is that something the user
is supposed to type?

> +The standard keybindings in @code{macrostep-mode} are the following:
> +@table @kbd
> +@item e
> +@itemx =
> +@itemx RET
> +@findex macrostep-expand

Please move the index entries to _before_ the corresponding @item's,
so that following the index entry lands you on the @item, not after
it.

Otherwise, this LGTM, thanks.

Re: Incorporate package macrostep into Emacs core

[Jeremy Bryant]

[-- Attachment #1: Type: text/plain, Size: 1680 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Jeremy Bryant <jb@jeremybryant.net>
>> Cc: monnier@iro.umontreal.ca,  emacs-devel@gnu.org
>> Date: Thu, 25 Apr 2024 22:27:56 +0100
>> 
>> Absolutely, sorry, attached is the revised patch I have worked on.
>
> Thanks, a few minor comments below.
>
>> Subject: [PATCH] Add Macrostep section in manual
>> 
>> Macrostep is an Emacs Lisp macro-expander useful for exploring and
>> writing macros.  This is based on Jonathan's Oddie's documentation
>> in the macrostep package.
>> 
>> 	* doc/emacs/programs.texi (Programs): Add Macrostep section.
>> 	* doc/emacs/emacs.texi (Top): 	Add detailed menu reference.
>  ^^^^^^^                             ^^^
> Those TABs should not be there in the commit log message.
>
>> +You can use @code{macrostep-mode} to explore Lisp macro definitions, and
>> +help write new macros, using @kbd{M-x macrostep-expand} when point is
>> +over a macro.
>
> We say "when point is in a macro".  "Over" is reserved for mouse
> pointer.
>
>> +@kbd{macrostep} is a minor mode for interactively stepping through the
>         ^^^^^^^^^
> "macrostep-mode", I guess?  And why @kbd? is that something the user
> is supposed to type?
>
>> +The standard keybindings in @code{macrostep-mode} are the following:
>> +@table @kbd
>> +@item e
>> +@itemx =
>> +@itemx RET
>> +@findex macrostep-expand
>
> Please move the index entries to _before_ the corresponding @item's,
> so that following the index entry lands you on the @item, not after
> it.
>
> Otherwise, this LGTM, thanks.

OK - Attached is a revised patch incorporating the ideas above (as well as a
refinement from Eshel on clarifying this is for Emacs Lisp).


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-Macrostep-section-in-manual.patch --]
[-- Type: text/x-diff, Size: 3743 bytes --]

From 4be7abaa0af33936b62119732609a1d4ff8ee1b4 Mon Sep 17 00:00:00 2001
From: Jeremy Bryant <jb@jeremybryant.net>
Date: Tue, 23 Apr 2024 22:21:07 +0100
Subject: [PATCH] Add Macrostep section in manual

Macrostep is an Emacs Lisp macro-expander useful for exploring and
writing macros.  This is based on Jonathan's Oddie's documentation
in the macrostep package.

* doc/emacs/programs.texi (Programs): Add Macrostep section.
* doc/emacs/emacs.texi (Top): Add detailed menu reference.
---
 doc/emacs/emacs.texi    |  3 ++
 doc/emacs/programs.texi | 61 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 64 insertions(+)

diff --git a/doc/emacs/emacs.texi b/doc/emacs/emacs.texi
index 7d77f13ab21..2ce47dd6e5e 100644
--- a/doc/emacs/emacs.texi
+++ b/doc/emacs/emacs.texi
@@ -694,6 +694,9 @@ Top
 @ifnottex
 * Fortran::             Fortran mode and its special features.
 @end ifnottex
+@ifnottex
+* Macrostep::           Interactive Emacs Lisp macro-expander.
+@end ifnottex
 
 Top-Level Definitions, or Defuns
 
diff --git a/doc/emacs/programs.texi b/doc/emacs/programs.texi
index de28a9f1dd4..8c8542cc374 100644
--- a/doc/emacs/programs.texi
+++ b/doc/emacs/programs.texi
@@ -45,6 +45,10 @@ Programs
 @ifnottex
 * Fortran::             Fortran mode and its special features.
 @end ifnottex
+@ifnottex
+* Macrostep::           Interactive Emacs Lisp macro-expander.
+@end ifnottex
+
 @end menu
 
 @node Program Modes
@@ -2235,3 +2239,60 @@ Asm Mode
 @ifnottex
 @include fortran-xtra.texi
 @end ifnottex
+
+@ifnottex
+@node Macrostep
+@section Macrostep: interactive Emacs Lisp macro-expander
+
+You can use @code{macrostep-mode} to explore Lisp macro definitions, and
+help write new macros, using @kbd{M-x macrostep-expand} when point is
+in a macro.
+
+@code{macrostep-mode} is a minor mode for interactively stepping through the
+expansion of macros in Emacs Lisp source code.  It lets you see exactly
+what happens at each step of the expansion process by pretty-printing
+the expanded forms inline in the source buffer, which is temporarily
+read-only while macro expansions are visible.  You can expand and
+collapse macro forms one step at a time, and evaluate or instrument the
+expansions for debugging with Edebug as usual.  Single-stepping through
+the expansion is particularly useful for debugging macros that expand
+into another macro form.  These can be difficult to debug with
+@code{macroexpand}, which continues expansion until the top-level form
+is no longer a macro call.
+
+The standard keybindings in @code{macrostep-mode} are the following:
+@table @kbd
+@findex macrostep-expand
+@item e
+@itemx =
+@itemx RET
+Expand the macro form following point one step (@code{macrostep-expand}).
+@findex macrostep-collapse
+@item c
+@itemx u
+@itemx DEL
+Collapse the form following point (@code{macrostep-collapse}).
+@findex macrostep-next-macro
+@item n
+@itemx TAB
+Jump to the next macro form in the expansion (@code{macrostep-next-macro}).
+@findex macrostep-previous-macro
+@item p
+@itemx M-TAB
+Jump to the previous macro form in the expansion (@code{macrostep-previous-macro}).
+@findex macrostep-collapse-all
+@item q
+@itemx C-c C-c
+Collapse all expanded forms and exit macrostep-mode (@code{macrostep-collapse-all}).
+@end table
+
+It's not very useful to enable and disable macrostep-mode directly.
+Instead, bind @code{macrostep-expand} to a key in
+@code{emacs-lisp-mode-map}, for example @kbd{C-c e}.
+
+You can then enter @code{macrostep-mode} and expand a macro form
+completely by typing @kbd{C-c e e e@dots{}} as many times as necessary.
+
+Exit macrostep-mode by typing @kbd{q} or @kbd{C-c C-c}, or by
+successively typing @kbd{c} to collapse all surrounding expansions.
+@end ifnottex
-- 
2.42.0

Re: Incorporate package macrostep into Emacs core

[Eli Zaretskii]

> From: Jeremy Bryant <jb@jeremybryant.net>
> Cc: monnier@iro.umontreal.ca,  emacs-devel@gnu.org
> Date: Mon, 29 Apr 2024 22:38:27 +0100
> 
> OK - Attached is a revised patch incorporating the ideas above (as well as a
> refinement from Eshel on clarifying this is for Emacs Lisp).

LGTM, but I believe we are waiting for when macrostep is actually
added to Emacs, is that right?

Re: Incorporate package macrostep into Emacs core

[Jeremy Bryant]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Jeremy Bryant <jb@jeremybryant.net>
>> Cc: monnier@iro.umontreal.ca,  emacs-devel@gnu.org
>> Date: Mon, 29 Apr 2024 22:38:27 +0100
>> 
>> OK - Attached is a revised patch incorporating the ideas above (as well as a
>> refinement from Eshel on clarifying this is for Emacs Lisp).
>
> LGTM, but I believe we are waiting for when macrostep is actually
> added to Emacs, is that right?

Eli, thanks for reviewing, and good to know.

Indeed, this is pending inclusion of macrostep.

Re: Incorporate package macrostep into Emacs core

[Jeremy Bryant]

Jeremy Bryant <jb@jeremybryant.net> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: Jeremy Bryant <jb@jeremybryant.net>
>>> Cc: monnier@iro.umontreal.ca,  emacs-devel@gnu.org
>>> Date: Mon, 29 Apr 2024 22:38:27 +0100
>>> 
>>> OK - Attached is a revised patch incorporating the ideas above (as well as a
>>> refinement from Eshel on clarifying this is for Emacs Lisp).
>>
>> LGTM, but I believe we are waiting for when macrostep is actually
>> added to Emacs, is that right?
>
> Eli, thanks for reviewing, and good to know.
>
> Indeed, this is pending inclusion of macrostep.

Here is a quick status update (given interest from a question to Philip at
EmacsConf) on the potential inclusion of macrostep in Emacs.


macrostep author Jonathan Oddie is supportive of the copyright
assignment but there have been delays until May 2024, due to the
uniqueness of the party involved.  I'm following up on the latest
including with the FSF's position.

other contributors also completed FSF paperwork or their contributions deemed exempt.

the macrostep package available through NonGNU-ELPA is upstreamed at
Jonas's `orphanage' github, at
https://github.com/emacsorphanage/macrostep

For the time being I'm also a maintainer of this package, and created the
last 2 releases, 0.9.3 and 0.9.4.
That is where the changes suggested on this list have been done.

(the manual patch referred to above is kept aside for the eventual inclusion)

Re: Incorporate package macrostep into Emacs core

[Stefan Kangas]

Jeremy Bryant <jb@jeremybryant.net> writes:

> Here is a quick status update (given interest from a question to Philip at
> EmacsConf) on the potential inclusion of macrostep in Emacs.
>
> macrostep author Jonathan Oddie is supportive of the copyright
> assignment but there have been delays until May 2024, due to the
> uniqueness of the party involved.  I'm following up on the latest
> including with the FSF's position.
>
> other contributors also completed FSF paperwork or their contributions deemed exempt.

So, in summary, we are only waiting for one assignment to come through
before we can add it?  That is very good news.

Thanks again for working on this.