replacing ldefs-boot.el

replacing ldefs-boot.el

[Phillip Lord]

I've been working on a patch to replace ldefs-boot.el.

ldefs-boot.el is necessary for Emacs to build from temacs to bootstrap,
and works by providing all the autoloads necessary for Emacs to build to
bootstrap and then compile all the COMPILE_FIRST sources.

Currently, ldefs-boot.el is just a copy of loaddefs.el, copied into the
repo regularly. It's a large, generated file and has to be in the repo
(or else you would need a copy from somewhere else to build Emacs).

It works by providing a much smaller file which provides just the
autoloads necessary to build to bootstrap, and then generates
loaddefs.el before it COMPILE_FIRST's. This file can also be regenerated
automatically, although I suspect that this will be a rare occurence.

It's not 100% finished yet (some comments need cleaning up), but I'd
like in-principle agreement for this to go to master before I do that.

The code is on feature/manual-ldefs-rebased

Phil

Re: replacing ldefs-boot.el

[Eli Zaretskii]

> From: phillip.lord@russet.org.uk (Phillip Lord)
> Date: Sun, 20 Nov 2016 17:46:52 +0000
> 
> It works by providing a much smaller file which provides just the
> autoloads necessary to build to bootstrap, and then generates
> loaddefs.el before it COMPILE_FIRST's. This file can also be regenerated
> automatically, although I suspect that this will be a rare occurence.
> 
> It's not 100% finished yet (some comments need cleaning up), but I'd
> like in-principle agreement for this to go to master before I do that.

Thanks for working on this.

It isn't clear to me to which aspects would you like to have an
a-priori agreement.  Can you elaborate?

Re: replacing ldefs-boot.el

[Phillip Lord]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: phillip.lord@russet.org.uk (Phillip Lord)
>> Date: Sun, 20 Nov 2016 17:46:52 +0000
>> 
>> It works by providing a much smaller file which provides just the
>> autoloads necessary to build to bootstrap, and then generates
>> loaddefs.el before it COMPILE_FIRST's. This file can also be regenerated
>> automatically, although I suspect that this will be a rare occurence.
>> 
>> It's not 100% finished yet (some comments need cleaning up), but I'd
>> like in-principle agreement for this to go to master before I do that.
>
> Thanks for working on this.
>
> It isn't clear to me to which aspects would you like to have an
> a-priori agreement.  Can you elaborate?

Assuming I fix the comments, I wanted to add the whole patch.

This is a reasonable significant change in the bootstrap build, and
there may be implications that I have not foreseen. Hence it makes sense
to ask first.

Phil

Re: replacing ldefs-boot.el

[Eli Zaretskii]

> From: phillip.lord@russet.org.uk (Phillip Lord)
> Cc: emacs-devel@gnu.org
> Date: Sun, 20 Nov 2016 21:28:49 +0000
> 
> This is a reasonable significant change in the bootstrap build, and
> there may be implications that I have not foreseen. Hence it makes sense
> to ask first.

Then please describe the bootstrap and the normal build in detail,
because I'm not sure I understood the implications by just looking at
the diffs.  What are those additional files, and how and when they are
generated?

Thanks.

Re: replacing ldefs-boot.el

[Phillip Lord]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: phillip.lord@russet.org.uk (Phillip Lord)
>> Cc: emacs-devel@gnu.org
>> Date: Sun, 20 Nov 2016 21:28:49 +0000
>> 
>> This is a reasonable significant change in the bootstrap build, and
>> there may be implications that I have not foreseen. Hence it makes sense
>> to ask first.
>
> Then please describe the bootstrap and the normal build in detail,
> because I'm not sure I understood the implications by just looking at
> the diffs.

Currently, emacs builds like so (in f make syntax)-- 

temacs: *.c

bootstrap-emacs: temacs
      ## temacs loads ldefs-boot.el during loadup or it crashes

compile-first: bootstrap-emacs

loaddefs.el: compile-first
     ## now we generate loaddefs.el


Afterwards, we do:

temacs: *.c

bootstrap-emacs: temacs
      ## temacs loads ldefs-boot-manual.el and -auto.el during
      ## loadup or it crashes

loaddefs.el: bootstrap-emacs
     ## now we generate loaddefs.el

compile-first: loaddefs.el


So, loaddefs.el is available before compile-first happens.


> What are those additional files, and how and when they are
> generated?

ldefs-boot-auto.el is now autogenerated from a make target at top level.
It works but augmenting eval.c, so that when an autoload happens, Emacs
writes to system stdout the equivalent autoload form. A complete
bootstrap build will, therefore, print all the autoloads that are
actually used. There's some lisp in admin/ldefs-clean.el which removes
everything except these lisp forms.

Advantages:

 - we still have an autogenerated file in the repo, but it is very much
   smaller (160 lines vs 40k).

Disadvantages:

 - generating ldefs-boot-auto is a lot slower (i.e. cp loaddefs.el
   ldefs-boot.el vs a complete bootstrap build of Emacs).

Invariants:

 - to my testing there is negligable difference in build time.


Phil

Re: replacing ldefs-boot.el

[Phillip Lord]

phillip.lord@russet.org.uk (Phillip Lord) writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: phillip.lord@russet.org.uk (Phillip Lord)
>>> Cc: emacs-devel@gnu.org
>>> Date: Sun, 20 Nov 2016 21:28:49 +0000
>>> 
>>> This is a reasonable significant change in the bootstrap build, and
>>> there may be implications that I have not foreseen. Hence it makes sense
>>> to ask first.
>>
>> Then please describe the bootstrap and the normal build in detail,
>> because I'm not sure I understood the implications by just looking at
>> the diffs.
>
> Currently, emacs builds like so (in f make syntax)-- 
>
> temacs: *.c
>
> bootstrap-emacs: temacs
>       ## temacs loads ldefs-boot.el during loadup or it crashes
>
> compile-first: bootstrap-emacs
>
> loaddefs.el: compile-first
>      ## now we generate loaddefs.el
>
>
> Afterwards, we do:
>
> temacs: *.c
>
> bootstrap-emacs: temacs
>       ## temacs loads ldefs-boot-manual.el and -auto.el during
>       ## loadup or it crashes
>
> loaddefs.el: bootstrap-emacs
>      ## now we generate loaddefs.el
>
> compile-first: loaddefs.el
>
>
> So, loaddefs.el is available before compile-first happens.
>
>
>> What are those additional files, and how and when they are
>> generated?
>
> ldefs-boot-auto.el is now autogenerated from a make target at top level.
> It works but augmenting eval.c, so that when an autoload happens, Emacs
> writes to system stdout the equivalent autoload form. A complete
> bootstrap build will, therefore, print all the autoloads that are
> actually used. There's some lisp in admin/ldefs-clean.el which removes
> everything except these lisp forms.
>
> Advantages:
>
>  - we still have an autogenerated file in the repo, but it is very much
>    smaller (160 lines vs 40k).
>
> Disadvantages:
>
>  - generating ldefs-boot-auto is a lot slower (i.e. cp loaddefs.el
>    ldefs-boot.el vs a complete bootstrap build of Emacs).
>
> Invariants:
>
>  - to my testing there is negligable difference in build time.


Have you had a chance to look at this yet?

Phil

Re: replacing ldefs-boot.el

[Eli Zaretskii]

> From: phillip.lord@russet.org.uk (Phillip Lord)
> Date: Fri, 25 Nov 2016 11:23:25 +0000
> Cc: emacs-devel@gnu.org
> 
> Have you had a chance to look at this yet?

I looked at this when you first published the branch, so yes.

I hoped others will offer their opinions.  Not sure what does this
silence mean.

Re: replacing ldefs-boot.el

[Paul Eggert]

Eli Zaretskii wrote:
> Not sure what does this
> silence mean.

We have a long holiday weekend in the U.S., for what that's worth. I haven't 
reviewed the patch carefully, though I do like the idea of not putting 
automatically-generated files into the repository.

Re: replacing ldefs-boot.el

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Fri, 25 Nov 2016 10:10:15 -0800
> 
> Eli Zaretskii wrote:
> > Not sure what does this
> > silence mean.
> 
> We have a long holiday weekend in the U.S., for what that's worth.

And a still longer one after that ;-)

> I haven't reviewed the patch carefully, though I do like the idea of
> not putting automatically-generated files into the repository.

AFAIU, the suggested method still puts auto-generated files into the
repository.

Re: replacing ldefs-boot.el

[Stefan Monnier]

> AFAIU, the suggested method still puts auto-generated files into the
> repository.

That's right.  What it does is to strip down ldefs-boot.el to just the
autoloads actually used.

The intention is to reduce the rate at which this file changes, and hence
reducing the spurious conflicts such a file inevitably causes.


        Stefan

Re: replacing ldefs-boot.el

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Fri, 25 Nov 2016 15:11:39 -0500
> 
> > AFAIU, the suggested method still puts auto-generated files into the
> > repository.
> 
> That's right.  What it does is to strip down ldefs-boot.el to just the
> autoloads actually used.
> 
> The intention is to reduce the rate at which this file changes, and hence
> reducing the spurious conflicts such a file inevitably causes.

I never had any such conflicts.  And since Glenn set up an automated
job that auto-commits ldefs-boot.el, the problem with changes seems
all but resolved.

Re: replacing ldefs-boot.el

[Phillip Lord]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Stefan Monnier <monnier@iro.umontreal.ca>
>> Date: Fri, 25 Nov 2016 15:11:39 -0500
>> 
>> > AFAIU, the suggested method still puts auto-generated files into the
>> > repository.
>> 
>> That's right.  What it does is to strip down ldefs-boot.el to just the
>> autoloads actually used.
>> 
>> The intention is to reduce the rate at which this file changes, and hence
>> reducing the spurious conflicts such a file inevitably causes.
>
> I never had any such conflicts.  And since Glenn set up an automated
> job that auto-commits ldefs-boot.el, the problem with changes seems
> all but resolved.

ldefs-boot.el wont cause conflicts per se, because it is not generated
during the build, nor is it versioned -- the situation we had previously
with other autoload files. This change should mean that the necessity
for committing ldefs-boot.el is considerably reduced, since
ldefs-boot-auto.el depends on many fewer files than ldefs-boot.el, and
should mean that diffs are more accurate. It's a small advantage; not
having auto-generated files versioned is perhaps an obsession of mine.

The risk is that it will need more manual intervention. ldefs-boot-auto
is not a complete replacement for ldefs-boot.el. I only know which
autoloaded *functions* have been called. Other forms in autoload are not
captured and have been added manually

(defvar warning-series nil)
(defvar electric-pair-text-pairs '((34 . 34)))

both of which turn out (perhaps surprisingly) to be necessary for
bootstrap. My plan would be to code these dependencies out.

Phil

Re: replacing ldefs-boot.el

[Phillip Lord]

Eli Zaretskii <eliz@gnu.org> writes:

>> Cc: emacs-devel@gnu.org
>> From: Paul Eggert <eggert@cs.ucla.edu>
>> Date: Fri, 25 Nov 2016 10:10:15 -0800
>> 
>> Eli Zaretskii wrote:
>> > Not sure what does this
>> > silence mean.
>> 
>> We have a long holiday weekend in the U.S., for what that's worth.
>
> And a still longer one after that ;-)
>
>> I haven't reviewed the patch carefully, though I do like the idea of
>> not putting automatically-generated files into the repository.
>
> AFAIU, the suggested method still puts auto-generated files into the
> repository.


My strong suspicion is that, while ldefs-boot-auto.el is generated, it
will need to be regenerated very rarely. Of course, I have little basis
for this suspicion and it's pretty hard to test.

Phil

Re: replacing ldefs-boot.el

[Óscar Fuentes]

phillip.lord@russet.org.uk (Phillip Lord) writes:

> My strong suspicion is that, while ldefs-boot-auto.el is generated, it
> will need to be regenerated very rarely. Of course, I have little basis
> for this suspicion and it's pretty hard to test.

You can go backwards in time: checkout some past revision, apply your
patch, run the build and see if the file changes. Repeat.

Re: replacing ldefs-boot.el

[Phillip Lord]

Óscar Fuentes <ofv@wanadoo.es> writes:

> phillip.lord@russet.org.uk (Phillip Lord) writes:
>
>> My strong suspicion is that, while ldefs-boot-auto.el is generated, it
>> will need to be regenerated very rarely. Of course, I have little basis
>> for this suspicion and it's pretty hard to test.
>
> You can go backwards in time: checkout some past revision, apply your
> patch, run the build and see if the file changes. Repeat.

Yes, it is possible, but it requires a full, single threaded bootstrap
build, which takes about an hour for me.

The same ldefs-boot (generated from master) works on emacs-25 head, and
master. It fails on emacs24 head because of inline.el and emacs-24.3
because of nadvice.

Of course this is going backward in time, not forward, so the number of
required autoloads could just be getting bigger. But given that the
number of quite small, that seems unlikely.

Phil

Re: replacing ldefs-boot.el

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 299 bytes --]

Paul Eggert <eggert@cs.ucla.edu> writes:

> We have a long holiday weekend in the U.S., for what that's worth. I haven't 
> reviewed the patch carefully, though I do like the idea of not putting 
> automatically-generated files into the repository.

I like the idea as well.

Cheers,
Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 472 bytes --]

Re: replacing ldefs-boot.el

[Noam Postavsky]

On Mon, Nov 21, 2016 at 5:03 PM, Phillip Lord
<phillip.lord@russet.org.uk> wrote:
>
> ldefs-boot-auto.el is now autogenerated from a make target at top level.
> It works but augmenting eval.c, so that when an autoload happens, Emacs
> writes to system stdout the equivalent autoload form. A complete
> bootstrap build will, therefore, print all the autoloads that are
> actually used. There's some lisp in admin/ldefs-clean.el which removes
> everything except these lisp forms.
>
> Advantages:
>
>  - we still have an autogenerated file in the repo, but it is very much
>    smaller (160 lines vs 40k).
>
> Disadvantages:
>
>  - generating ldefs-boot-auto is a lot slower (i.e. cp loaddefs.el
>    ldefs-boot.el vs a complete bootstrap build of Emacs).
>

Is it possible another disadvantage would be that ldefs-boot-auto
requires regeneration more often (because the list of autoloads used
is more precise)?

I notice several almost duplicated lines in ldefs-boot-auto.el, like

(autoload 'byte-compile "bytecomp" nil nil nil)
(autoload 'byte-compile "bytecomp" nil nil t)

byte-compile isn't a macro; I think you should put your addition to
Fautoload_do_load after the "if (EQ (macro_only, Qmacro))" check to
avoid falsely identifying macros.

Instead of printing the autoload lines directly to stderr and then
cleaning up afterwards, wouldn't it be simpler to collect them into a
hashtable, and write to a file at the end?

And doesn't this change in Makefile.in break parallel 'make bootstrap'
invocations?

-bootstrap: bootstrap-clean
+bootstrap: bootstrap-clean bootstrap-build
+
+bootstrap-build:

Re: replacing ldefs-boot.el

[Phillip Lord]

Noam Postavsky <npostavs@users.sourceforge.net> writes:
>> Disadvantages:
>>
>>  - generating ldefs-boot-auto is a lot slower (i.e. cp loaddefs.el
>>    ldefs-boot.el vs a complete bootstrap build of Emacs).
>>
>
> Is it possible another disadvantage would be that ldefs-boot-auto
> requires regeneration more often (because the list of autoloads used
> is more precise)?


No, I don't think that this is the case. ldefs-boot.el is actually
regenerated regularly IIUC (I think Glenn does it on a cron job; or he
gets up specially at 6am on the first of every month). And it changes
each time since it reflects changes in any autoloads any where in Emacs
(nearly), including things like documentation changes.

I have no idea what percentage of those commits are actually necessary
-- the unfortunate reality is that you can only test this with a full
bootstrap build which is something that most developers do rarely.


> I notice several almost duplicated lines in ldefs-boot-auto.el, like
>
> (autoload 'byte-compile "bytecomp" nil nil nil)
> (autoload 'byte-compile "bytecomp" nil nil t)
>
> byte-compile isn't a macro; I think you should put your addition to
> Fautoload_do_load after the "if (EQ (macro_only, Qmacro))" check to
> avoid falsely identifying macros.

Yeah, that's a screw up, thanks for picking this up.

I don't understand the semantics of the macro_only parameter, at all,
nor what that if statement is doing, but I'll try this fix.

> Instead of printing the autoload lines directly to stderr and then
> cleaning up afterwards, wouldn't it be simpler to collect them into a
> hashtable, and write to a file at the end?

Unfortunately not, because we are collating data from multiple
invocations of Emacs -- or rather temacs and bootstrap-emacs. We have to
store data in environment (i.e. a file), not in Emacs.


> And doesn't this change in Makefile.in break parallel 'make bootstrap'
> invocations?
>
> -bootstrap: bootstrap-clean
> +bootstrap: bootstrap-clean bootstrap-build
> +
> +bootstrap-build:

Does it? bootstrap-clean and bootstrap-build cannot be run in parallel
anyway. If it does cause issues, though, happy to remove; I added this
because it was useful during my development of the patch, rather than
because it was necessary.

Phil

Re: replacing ldefs-boot.el

[Noam Postavsky]

On Fri, Nov 25, 2016 at 4:57 PM, Phillip Lord
<phillip.lord@russet.org.uk> wrote:
> Noam Postavsky <npostavs@users.sourceforge.net> writes:
>>> Disadvantages:
>>>
>>>  - generating ldefs-boot-auto is a lot slower (i.e. cp loaddefs.el
>>>    ldefs-boot.el vs a complete bootstrap build of Emacs).
>>>
>>
>> Is it possible another disadvantage would be that ldefs-boot-auto
>> requires regeneration more often (because the list of autoloads used
>> is more precise)?
>
>
> No, I don't think that this is the case. ldefs-boot.el is actually
> regenerated regularly IIUC (I think Glenn does it on a cron job; or he
> gets up specially at 6am on the first of every month). And it changes
> each time since it reflects changes in any autoloads any where in Emacs
> (nearly), including things like documentation changes.

Oh, I didn't notice the frequency, I somehow had the impression it
just done as needed.

>
> I have no idea what percentage of those commits are actually necessary
> -- the unfortunate reality is that you can only test this with a full
> bootstrap build which is something that most developers do rarely.

My guess is that with the smaller ldefs-boot-auto it will be necessary
more often. But maybe not enough to matter, hard to say.

>
>
>> I notice several almost duplicated lines in ldefs-boot-auto.el, like
>>
>> (autoload 'byte-compile "bytecomp" nil nil nil)
>> (autoload 'byte-compile "bytecomp" nil nil t)
>>
>> byte-compile isn't a macro; I think you should put your addition to
>> Fautoload_do_load after the "if (EQ (macro_only, Qmacro))" check to
>> avoid falsely identifying macros.
>
> Yeah, that's a screw up, thanks for picking this up.
>
> I don't understand the semantics of the macro_only parameter, at all,
> nor what that if statement is doing, but I'll try this fix.

I think the value of FUNDEF tells you whether it's a macro, the
macro_only parameter just says to do nothing for non-macros.

>
>> And doesn't this change in Makefile.in break parallel 'make bootstrap'
>> invocations?
>>
>> -bootstrap: bootstrap-clean
>> +bootstrap: bootstrap-clean bootstrap-build
>> +
>> +bootstrap-build:
>
> Does it? bootstrap-clean and bootstrap-build cannot be run in parallel
> anyway.

Right. And I think the definitions you put would let make run them in
parallel, thus breaking things.

Re: replacing ldefs-boot.el

[Phillip Lord]

Noam Postavsky <npostavs@users.sourceforge.net> writes:
>> I have no idea what percentage of those commits are actually necessary
>> -- the unfortunate reality is that you can only test this with a full
>> bootstrap build which is something that most developers do rarely.
>
> My guess is that with the smaller ldefs-boot-auto it will be necessary
> more often. But maybe not enough to matter, hard to say.

Actually, it should be *necessary* exactly as often as now. We need a
certain number of autoloads to bootstrap. Both ldefs-boot-auto and
ldefs-boot provide these. But ldefs-boot provides many more as well.

The differnce is that when regenerating ldefs-boot-auto.el, we (should)
only get changes that really are necessary.


>>> I notice several almost duplicated lines in ldefs-boot-auto.el, like
>>>
>>> (autoload 'byte-compile "bytecomp" nil nil nil)
>>> (autoload 'byte-compile "bytecomp" nil nil t)
>>>
>>> byte-compile isn't a macro; I think you should put your addition to
>>> Fautoload_do_load after the "if (EQ (macro_only, Qmacro))" check to
>>> avoid falsely identifying macros.
>>
>> Yeah, that's a screw up, thanks for picking this up.
>>
>> I don't understand the semantics of the macro_only parameter, at all,
>> nor what that if statement is doing, but I'll try this fix.
>
> I think the value of FUNDEF tells you whether it's a macro, the
> macro_only parameter just says to do nothing for non-macros.

I changed my condition to this:

      Lisp_Object kind = Fnth (make_number (4), fundef);
      if (! (EQ (kind, Qt) || EQ (kind, Qmacro)))
        {
          fprintf(stderr, "nil");
        }
      else
        {
          fprintf(stderr, "t");
        }


which seems to work.


>>
>>> And doesn't this change in Makefile.in break parallel 'make bootstrap'
>>> invocations?
>>>
>>> -bootstrap: bootstrap-clean
>>> +bootstrap: bootstrap-clean bootstrap-build
>>> +
>>> +bootstrap-build:
>>
>> Does it? bootstrap-clean and bootstrap-build cannot be run in parallel
>> anyway.
>
> Right. And I think the definitions you put would let make run them in
> parallel, thus breaking things.


bootstrap: | bootstrap-clean bootstrap-build

?

Phil

Re: replacing ldefs-boot.el

[Noam Postavsky]

On Sat, Nov 26, 2016 at 7:19 AM, Phillip Lord
<phillip.lord@russet.org.uk> wrote:
> Noam Postavsky <npostavs@users.sourceforge.net> writes:
>>> I have no idea what percentage of those commits are actually necessary
>>> -- the unfortunate reality is that you can only test this with a full
>>> bootstrap build which is something that most developers do rarely.
>>
>> My guess is that with the smaller ldefs-boot-auto it will be necessary
>> more often. But maybe not enough to matter, hard to say.
>
> Actually, it should be *necessary* exactly as often as now. We need a
> certain number of autoloads to bootstrap. Both ldefs-boot-auto and
> ldefs-boot provide these. But ldefs-boot provides many more as well.

Yes, and sometimes the extra ones that ldefs-boot provides could
become necessary. In that case there wouldn't be a need to regenerate
because they're already there. When using ldefs-boot-auto we would
need to regenerate.

Does that sound right?

>>> Does it? bootstrap-clean and bootstrap-build cannot be run in parallel
>>> anyway.
>>
>> Right. And I think the definitions you put would let make run them in
>> parallel, thus breaking things.
>
>
> bootstrap: | bootstrap-clean bootstrap-build
>
> ?

I think it should be

bootstrap: bootstrap-clean bootstrap-build
boostrap-build: | bootstrap-clean

Re: replacing ldefs-boot.el

[Phillip Lord]

Noam Postavsky <npostavs@users.sourceforge.net> writes:
>> Actually, it should be *necessary* exactly as often as now. We need a
>> certain number of autoloads to bootstrap. Both ldefs-boot-auto and
>> ldefs-boot provide these. But ldefs-boot provides many more as well.
>
> Yes, and sometimes the extra ones that ldefs-boot provides could
> become necessary. In that case there wouldn't be a need to regenerate
> because they're already there. When using ldefs-boot-auto we would
> need to regenerate.
>
> Does that sound right?

Yes, that's true. This might occur, for example, if a file is re-ordered
so that a new function is hit first.

My suspision is that once we have a clear idea of the functions that are
used during bootstrap, this would happen rarely.

>>> Right. And I think the definitions you put would let make run them in
>>> parallel, thus breaking things.
>>
>>
>> bootstrap: | bootstrap-clean bootstrap-build
>>
>> ?
>
> I think it should be
>
> bootstrap: bootstrap-clean bootstrap-build
> boostrap-build: | bootstrap-clean

Ta!

Phil

Re: replacing ldefs-boot.el

[Phillip Lord]

phillip.lord@russet.org.uk (Phillip Lord) writes:
>
> Advantages:
>
>  - we still have an autogenerated file in the repo, but it is very much
>    smaller (160 lines vs 40k).
>
> Disadvantages:
>
>  - generating ldefs-boot-auto is a lot slower (i.e. cp loaddefs.el
>    ldefs-boot.el vs a complete bootstrap build of Emacs).
>
> Invariants:
>
>  - to my testing there is negligable difference in build time.

I've added another commit to this now which hopefully fixes the
references to ldefs-boot.el, replacing them or generalizing them as
appropriate.

update_autogen has not been done, and I am unsure about it. The main
disadvantage with my patch is that it needs a full bootstrap to
regenerate. Do we want this in update_autogen or not?

Re: replacing ldefs-boot.el

[Phillip Lord]

phillip.lord@russet.org.uk (Phillip Lord) writes:

> I've been working on a patch to replace ldefs-boot.el.
>

> It's not 100% finished yet (some comments need cleaning up), but I'd
> like in-principle agreement for this to go to master before I do that.
>
> The code is on feature/manual-ldefs-rebased



Just revisting this; I believe this code is complete now, and I'd like
to merge to master (after squashing).

John, Eli are you happy?

Phil

Re: replacing ldefs-boot.el

[Noam Postavsky]

On Tue, Dec 13, 2016 at 7:17 AM, Phillip Lord
<phillip.lord@russet.org.uk> wrote:
>
> Just revisting this; I believe this code is complete now, and I'd like
> to merge to master (after squashing).

Oh, that reminds me, I had wanted to ask if a *full* bootstrap was
really necessary for running admin/ldefs-clean.el, wouldn't the
bootstrap-emacs be enough?

Re: replacing ldefs-boot.el

[Phillip Lord]

Noam Postavsky <npostavs@users.sourceforge.net> writes:

> On Tue, Dec 13, 2016 at 7:17 AM, Phillip Lord
> <phillip.lord@russet.org.uk> wrote:
>>
>> Just revisting this; I believe this code is complete now, and I'd like
>> to merge to master (after squashing).
>
> Oh, that reminds me, I had wanted to ask if a *full* bootstrap was
> really necessary for running admin/ldefs-clean.el, wouldn't the
> bootstrap-emacs be enough?

That's possible. Likewise generate-ldefs-boot depends on "all" because
it needs a new loaddefs.el to bootstrap while generating
ldefs-boot-auto. Probably building to bootstrap-emacs is enough for the
former, and loaddefs.el is enough for the later.

The reason that I didn't do this is because there are no top-level
targets for building to those levels; using recursive make calls to src
and lisp seemed fragile to me, and a gives a performance gain for
something that is likely to only be run rarely.

Phil

Re: replacing ldefs-boot.el

[John Wiegley]

[-- Attachment #1: Type: text/plain, Size: 426 bytes --]

>>>>> "PL" == Phillip Lord <phillip.lord@russet.org.uk> writes:

PL> Just revisting this; I believe this code is complete now, and I'd like to
PL> merge to master (after squashing).

PL> John, Eli are you happy?

Yes, this looks like a nice simplification to me.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 658 bytes --]

Re: replacing ldefs-boot.el

[Eli Zaretskii]

> From: phillip.lord@russet.org.uk (Phillip Lord)
> Date: Tue, 13 Dec 2016 12:17:47 +0000
> 
> Just revisting this; I believe this code is complete now, and I'd like
> to merge to master (after squashing).
> 
> John, Eli are you happy?

I'm not sure people were enthusiastic about it.  But I'm not opposed,
either.

Re: replacing ldefs-boot.el

[Phillip Lord]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: phillip.lord@russet.org.uk (Phillip Lord)
>> Date: Tue, 13 Dec 2016 12:17:47 +0000
>> 
>> Just revisting this; I believe this code is complete now, and I'd like
>> to merge to master (after squashing).
>> 
>> John, Eli are you happy?
>
> I'm not sure people were enthusiastic about it.  But I'm not opposed,
> either.

I think several people liked getting rid of the autogen file (or at
least making it smaller); you didn't think this was an issue. Several
specific potential issues were raised, but no one objected in principle.

I've added it now. If it causes problems in practice, I will back it out
and think again.

Phil