* Rewriting make-docfile.c in Lisp?
@ 2021-05-05 15:56 sbaugh
2021-05-05 16:13 ` Eli Zaretskii
0 siblings, 1 reply; 16+ messages in thread
From: sbaugh @ 2021-05-05 15:56 UTC (permalink / raw)
To: emacs-devel
I was recently trawling through make-docfile.c and wondered:
Couldn't this be written in Lisp?
It seems like this complex parsing code would be much simpler in Lisp,
and there's no reason the make-docfile step couldn't run after Emacs is
built. It would be a fair bit of work, but I think the resulting
make-docfile.el would be much smaller and easier to understand.
I'm not about to go do it, but it would be a good self-contained
project, I think, so maybe a good addition to TODO?
Does anyone have any objections or reasons why this shouldn't be done?
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Rewriting make-docfile.c in Lisp?
2021-05-05 15:56 Rewriting make-docfile.c in Lisp? sbaugh
@ 2021-05-05 16:13 ` Eli Zaretskii
2021-05-05 16:49 ` Spencer Baugh
0 siblings, 1 reply; 16+ messages in thread
From: Eli Zaretskii @ 2021-05-05 16:13 UTC (permalink / raw)
To: sbaugh; +Cc: emacs-devel
> From: sbaugh@catern.com
> Date: Wed, 05 May 2021 11:56:03 -0400
>
> I was recently trawling through make-docfile.c and wondered:
>
> Couldn't this be written in Lisp?
make-docfile does several jobs; did you mean to convert them all to
Lisp? One of them is globals.h, which is needed for compiling the
Emacs C sources.
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Rewriting make-docfile.c in Lisp?
2021-05-05 16:13 ` Eli Zaretskii
@ 2021-05-05 16:49 ` Spencer Baugh
2021-05-05 17:16 ` Alan Mackenzie
2021-05-05 18:49 ` Eli Zaretskii
0 siblings, 2 replies; 16+ messages in thread
From: Spencer Baugh @ 2021-05-05 16:49 UTC (permalink / raw)
To: Eli Zaretskii; +Cc: emacs-devel
Eli Zaretskii <eliz@gnu.org> writes:
>> Couldn't this be written in Lisp?
>
> make-docfile does several jobs; did you mean to convert them all to
> Lisp? One of them is globals.h, which is needed for compiling the
> Emacs C sources.
Ah, I didn't see that globals.h was generated by make-docfile. I just
meant generating etc/DOC. Pluss any other jobs done by make-docfile that
aren't required to compile Emacs, if there are any.
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Rewriting make-docfile.c in Lisp?
2021-05-05 16:49 ` Spencer Baugh
@ 2021-05-05 17:16 ` Alan Mackenzie
2021-05-05 18:39 ` Spencer Baugh
2021-05-05 18:49 ` Eli Zaretskii
1 sibling, 1 reply; 16+ messages in thread
From: Alan Mackenzie @ 2021-05-05 17:16 UTC (permalink / raw)
To: Spencer Baugh; +Cc: Eli Zaretskii, emacs-devel
Hello, Spencer.
On Wed, May 05, 2021 at 12:49:12 -0400, Spencer Baugh wrote:
> Eli Zaretskii <eliz@gnu.org> writes:
> >> Couldn't this be written in Lisp?
> >
> > make-docfile does several jobs; did you mean to convert them all to
> > Lisp? One of them is globals.h, which is needed for compiling the
> > Emacs C sources.
> Ah, I didn't see that globals.h was generated by make-docfile. I just
> meant generating etc/DOC. Pluss any other jobs done by make-docfile that
> aren't required to compile Emacs, if there are any.
Why would one want to write it in Lisp anyway?
We have a working program in C which requires little maintenance, yet
which is used continually. Rewriting it in Lisp would be _work_.
Also, the Lisp version would run more slowly than the C version, leading
to more irritation over build speeds than there currently is. I think
this is also part of the build which holds things up in a single core,
thus making its speed more important than, say, a C or Lisp compilation.
--
Alan Mackenzie (Nuremberg, Germany).
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Rewriting make-docfile.c in Lisp?
2021-05-05 17:16 ` Alan Mackenzie
@ 2021-05-05 18:39 ` Spencer Baugh
2021-05-05 19:08 ` Stefan Monnier
2021-05-05 19:10 ` Eli Zaretskii
0 siblings, 2 replies; 16+ messages in thread
From: Spencer Baugh @ 2021-05-05 18:39 UTC (permalink / raw)
To: Alan Mackenzie; +Cc: Eli Zaretskii, emacs-devel
Alan Mackenzie <acm@muc.de> writes:
> Why would one want to write it in Lisp anyway?
>
> We have a working program in C which requires little maintenance, yet
> which is used continually. Rewriting it in Lisp would be _work_.
I was making a change and found it quite difficult. I think it would be
much simpler to change if it was in Lisp. I admit that since it's
rarely changed, it may not be worth it.
> Also, the Lisp version would run more slowly than the C version, leading
> to more irritation over build speeds than there currently is. I think
> this is also part of the build which holds things up in a single core,
> thus making its speed more important than, say, a C or Lisp compilation.
A rewrite in Lisp could also take the opportunity to add parallelization
and speed it up, if indeed this is a bottleneck for the build.
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Rewriting make-docfile.c in Lisp?
2021-05-05 16:49 ` Spencer Baugh
2021-05-05 17:16 ` Alan Mackenzie
@ 2021-05-05 18:49 ` Eli Zaretskii
1 sibling, 0 replies; 16+ messages in thread
From: Eli Zaretskii @ 2021-05-05 18:49 UTC (permalink / raw)
To: Spencer Baugh; +Cc: emacs-devel
> From: Spencer Baugh <sbaugh@catern.com>
> Cc: emacs-devel@gnu.org
> Date: Wed, 05 May 2021 12:49:12 -0400
>
> Eli Zaretskii <eliz@gnu.org> writes:
> >> Couldn't this be written in Lisp?
> >
> > make-docfile does several jobs; did you mean to convert them all to
> > Lisp? One of them is globals.h, which is needed for compiling the
> > Emacs C sources.
>
> Ah, I didn't see that globals.h was generated by make-docfile. I just
> meant generating etc/DOC. Pluss any other jobs done by make-docfile that
> aren't required to compile Emacs, if there are any.
But writing globals.h uses the same text-parsing code as the other
jobs, so I'm not sure it's a win to rewrite only the DOC parts in
Lisp, as we'd still need to maintain the globals part.
Does anyone else have an opinion?
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Rewriting make-docfile.c in Lisp?
2021-05-05 18:39 ` Spencer Baugh
@ 2021-05-05 19:08 ` Stefan Monnier
2021-05-05 19:13 ` Eli Zaretskii
` (2 more replies)
2021-05-05 19:10 ` Eli Zaretskii
1 sibling, 3 replies; 16+ messages in thread
From: Stefan Monnier @ 2021-05-05 19:08 UTC (permalink / raw)
To: Spencer Baugh; +Cc: Alan Mackenzie, Eli Zaretskii, emacs-devel
>> Why would one want to write it in Lisp anyway?
>> We have a working program in C which requires little maintenance, yet
>> which is used continually. Rewriting it in Lisp would be _work_.
Agreed: I'm generally all for writing ELisp instead of C, but
I wouldn't bother rewriting this code in ELisp, personally.
> I was making a change and found it quite difficult. I think it would
> be much simpler to change if it was in Lisp. I admit that since it's
> rarely changed, it may not be worth it.
Maybe if you described the change that motivates your proposal, we'd be
less negative ;-)
>> Also, the Lisp version would run more slowly than the C version, leading
>> to more irritation over build speeds than there currently is. I think
>> this is also part of the build which holds things up in a single core,
>> thus making its speed more important than, say, a C or Lisp compilation.
> A rewrite in Lisp could also take the opportunity to add parallelization
> and speed it up, if indeed this is a bottleneck for the build.
I don't think it can be sped up by writing it in ELisp (on the contrary).
It could be changed such that `make` can run it in parallel on each
C file, indeed, but it would have to be done well enough to make up for
the slowdown imposed by the ELisp version.
Stefan
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Rewriting make-docfile.c in Lisp?
2021-05-05 18:39 ` Spencer Baugh
2021-05-05 19:08 ` Stefan Monnier
@ 2021-05-05 19:10 ` Eli Zaretskii
2021-05-05 21:27 ` Spencer Baugh
1 sibling, 1 reply; 16+ messages in thread
From: Eli Zaretskii @ 2021-05-05 19:10 UTC (permalink / raw)
To: Spencer Baugh; +Cc: acm, emacs-devel
> From: Spencer Baugh <sbaugh@catern.com>
> Date: Wed, 05 May 2021 14:39:18 -0400
> Cc: Eli Zaretskii <eliz@gnu.org>, emacs-devel@gnu.org
>
> A rewrite in Lisp could also take the opportunity to add parallelization
> and speed it up, if indeed this is a bottleneck for the build.
It isn't a bottleneck in its current form, AFAIK, but what did you
mean by "parallelization" in this context?
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Rewriting make-docfile.c in Lisp?
2021-05-05 19:08 ` Stefan Monnier
@ 2021-05-05 19:13 ` Eli Zaretskii
2021-05-05 19:38 ` Stefan Monnier
2021-05-05 20:05 ` Glenn Morris
2021-05-05 21:11 ` Spencer Baugh
2 siblings, 1 reply; 16+ messages in thread
From: Eli Zaretskii @ 2021-05-05 19:13 UTC (permalink / raw)
To: Stefan Monnier; +Cc: sbaugh, emacs-devel, acm
> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Alan Mackenzie <acm@muc.de>, Eli Zaretskii <eliz@gnu.org>,
> emacs-devel@gnu.org
> Date: Wed, 05 May 2021 15:08:05 -0400
>
> It could be changed such that `make` can run it in parallel on each
> C file, indeed
But the same could be done with the C version, no?
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Rewriting make-docfile.c in Lisp?
2021-05-05 19:13 ` Eli Zaretskii
@ 2021-05-05 19:38 ` Stefan Monnier
0 siblings, 0 replies; 16+ messages in thread
From: Stefan Monnier @ 2021-05-05 19:38 UTC (permalink / raw)
To: Eli Zaretskii; +Cc: sbaugh, acm, emacs-devel
>> It could be changed such that `make` can run it in parallel on each
>> C file, indeed
>
> But the same could be done with the C version, no?
Of course (I think the argument was just that it might be easier to
make such a change to the code if it's written in ELisp than in C;
I don't know whether that's indeed the case, I haven't thought about it
nor refreshed my memory of how `make-docfile.c` currently works).
Stefan
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Rewriting make-docfile.c in Lisp?
2021-05-05 19:08 ` Stefan Monnier
2021-05-05 19:13 ` Eli Zaretskii
@ 2021-05-05 20:05 ` Glenn Morris
2021-05-05 21:18 ` Stefan Monnier
2021-05-05 21:11 ` Spencer Baugh
2 siblings, 1 reply; 16+ messages in thread
From: Glenn Morris @ 2021-05-05 20:05 UTC (permalink / raw)
To: Stefan Monnier; +Cc: Spencer Baugh, Eli Zaretskii, emacs-devel, Alan Mackenzie
> Agreed: I'm generally all for writing ELisp instead of C, but
> I wouldn't bother rewriting this code in ELisp, personally.
Sounds like the etc/TODO entry that suggests doing just this should be
updated/removed then. It refers to
https://lists.gnu.org/r/emacs-devel/2012-06/msg00037.html
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Rewriting make-docfile.c in Lisp?
2021-05-05 19:08 ` Stefan Monnier
2021-05-05 19:13 ` Eli Zaretskii
2021-05-05 20:05 ` Glenn Morris
@ 2021-05-05 21:11 ` Spencer Baugh
2021-05-06 8:45 ` Eli Zaretskii
2 siblings, 1 reply; 16+ messages in thread
From: Spencer Baugh @ 2021-05-05 21:11 UTC (permalink / raw)
To: Stefan Monnier; +Cc: Alan Mackenzie, Eli Zaretskii, emacs-devel
Stefan Monnier <monnier@iro.umontreal.ca> writes:
> Maybe if you described the change that motivates your proposal, we'd be
> less negative ;-)
Perhaps foolishly, I (experimentally) switched some existing uses of
DEFVAR_PER_BUFFER to a new variant with a new name, and then noticed
some test failures. I tried to grep for "DEFVAR_PER_BUFFER" (and other
variations) to find what might be depending on the name of the macro but
didn't find anything, since make-docfile.c parses DEFVAR_PER_BUFFER one
character at a time with getc and so doesn't actually include the string
"DEFVAR_PER_BUFFER" anywhere. Eventually I figured out that it was
make-docfile that wasn't picking up the new name for the
DEFVAR_PER_BUFFER macro. So then I dug through scan_c_stream in
make-docfile.c to see what would be required to add support for the
DEFVAR_PER_BUFFER variant, but eventually gave up on understanding the
best way to inject my new macro name into the manually-unrolled tree of
getc-based parsing.
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Rewriting make-docfile.c in Lisp?
2021-05-05 20:05 ` Glenn Morris
@ 2021-05-05 21:18 ` Stefan Monnier
2021-05-05 21:34 ` Stefan Monnier
0 siblings, 1 reply; 16+ messages in thread
From: Stefan Monnier @ 2021-05-05 21:18 UTC (permalink / raw)
To: Glenn Morris; +Cc: Spencer Baugh, Alan Mackenzie, Eli Zaretskii, emacs-devel
>> Agreed: I'm generally all for writing ELisp instead of C, but
>> I wouldn't bother rewriting this code in ELisp, personally.
> Sounds like the etc/TODO entry that suggests doing just this should be
> updated/removed then. It refers to
> https://lists.gnu.org/r/emacs-devel/2012-06/msg00037.html
But there the idea was specifically for the scan of .el(c) files and not
for the scan of C files. Also the idea was not to implement a new
scanning code but to do it from existing code that already looks at
the files (e.g. `autoloads.el` or the `bytecomp.el`).
Stefan
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Rewriting make-docfile.c in Lisp?
2021-05-05 19:10 ` Eli Zaretskii
@ 2021-05-05 21:27 ` Spencer Baugh
0 siblings, 0 replies; 16+ messages in thread
From: Spencer Baugh @ 2021-05-05 21:27 UTC (permalink / raw)
To: Eli Zaretskii; +Cc: acm, emacs-devel
Eli Zaretskii <eliz@gnu.org> writes:
>> From: Spencer Baugh <sbaugh@catern.com>
>> Date: Wed, 05 May 2021 14:39:18 -0400
>> Cc: Eli Zaretskii <eliz@gnu.org>, emacs-devel@gnu.org
>>
>> A rewrite in Lisp could also take the opportunity to add parallelization
>> and speed it up, if indeed this is a bottleneck for the build.
>
> It isn't a bottleneck in its current form, AFAIK, but what did you
> mean by "parallelization" in this context?
I meant to run it in parallel on each individual file, one Emacs process
running make-docfile.el per file; and then merge the results together at
the end in some fashion.
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Rewriting make-docfile.c in Lisp?
2021-05-05 21:18 ` Stefan Monnier
@ 2021-05-05 21:34 ` Stefan Monnier
0 siblings, 0 replies; 16+ messages in thread
From: Stefan Monnier @ 2021-05-05 21:34 UTC (permalink / raw)
To: Glenn Morris; +Cc: Spencer Baugh, Alan Mackenzie, Eli Zaretskii, emacs-devel
>>> Agreed: I'm generally all for writing ELisp instead of C, but
>>> I wouldn't bother rewriting this code in ELisp, personally.
>> Sounds like the etc/TODO entry that suggests doing just this should be
>> updated/removed then. It refers to
>> https://lists.gnu.org/r/emacs-devel/2012-06/msg00037.html
> But there the idea was specifically for the scan of .el(c) files and not
> for the scan of C files. Also the idea was not to implement a new
> scanning code but to do it from existing code that already looks at
> the files (e.g. `autoloads.el` or the `bytecomp.el`).
BTW, personally, I think a better goal would be to try and eliminate the
DOC file altogether: for the docstrings coming from C code we should be
able to keep them in normal static variables (and move them to
a separate section with `_attribute__ ((section (".docstrings")))` or
something like that), and for the docstrings coming from preloaded ELisp
files we can fetch them from the .elc file like we do for
non-preloaded files.
The `make-docfile.c` would still be needed (probably under a new name)
for things like `global.h`, of course, and I don't see a convenient way
to replace it with a version written in ELisp for bootstrapping reasons.
Stefan
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Rewriting make-docfile.c in Lisp?
2021-05-05 21:11 ` Spencer Baugh
@ 2021-05-06 8:45 ` Eli Zaretskii
0 siblings, 0 replies; 16+ messages in thread
From: Eli Zaretskii @ 2021-05-06 8:45 UTC (permalink / raw)
To: Spencer Baugh; +Cc: acm, monnier, emacs-devel
> From: Spencer Baugh <sbaugh@catern.com>
> Cc: Alan Mackenzie <acm@muc.de>, Eli Zaretskii <eliz@gnu.org>,
> emacs-devel@gnu.org
> Date: Wed, 05 May 2021 17:11:08 -0400
>
> Perhaps foolishly, I (experimentally) switched some existing uses of
> DEFVAR_PER_BUFFER to a new variant with a new name, and then noticed
> some test failures. I tried to grep for "DEFVAR_PER_BUFFER" (and other
> variations) to find what might be depending on the name of the macro but
> didn't find anything, since make-docfile.c parses DEFVAR_PER_BUFFER one
> character at a time with getc and so doesn't actually include the string
> "DEFVAR_PER_BUFFER" anywhere.
We could improve discoverability of this by adding appropriate
comments to the source code there.
> Eventually I figured out that it was make-docfile that wasn't
> picking up the new name for the DEFVAR_PER_BUFFER macro. So then I
> dug through scan_c_stream in make-docfile.c to see what would be
> required to add support for the DEFVAR_PER_BUFFER variant, but
> eventually gave up on understanding the best way to inject my new
> macro name into the manually-unrolled tree of getc-based parsing.
It's just a series of if/else if tests that examine the characters one
by one, so adding a new macro would mean some more tests like that.
Or am I missing something?
^ permalink raw reply [flat|nested] 16+ messages in thread
end of thread, other threads:[~2021-05-06 8:45 UTC | newest]
Thread overview: 16+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2021-05-05 15:56 Rewriting make-docfile.c in Lisp? sbaugh
2021-05-05 16:13 ` Eli Zaretskii
2021-05-05 16:49 ` Spencer Baugh
2021-05-05 17:16 ` Alan Mackenzie
2021-05-05 18:39 ` Spencer Baugh
2021-05-05 19:08 ` Stefan Monnier
2021-05-05 19:13 ` Eli Zaretskii
2021-05-05 19:38 ` Stefan Monnier
2021-05-05 20:05 ` Glenn Morris
2021-05-05 21:18 ` Stefan Monnier
2021-05-05 21:34 ` Stefan Monnier
2021-05-05 21:11 ` Spencer Baugh
2021-05-06 8:45 ` Eli Zaretskii
2021-05-05 19:10 ` Eli Zaretskii
2021-05-05 21:27 ` Spencer Baugh
2021-05-05 18:49 ` Eli Zaretskii
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/emacs.git
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.