Preloading seq.el

Preloading seq.el

[Stefan Kangas]

Hi,

I'm working on some things in help.el, and it would be very nice to be
able to use a function in seq.el to simplify my code.  Alas, seq.el is
not preloaded.

Could we preload seq.el?

Previous discussion here:
https://lists.gnu.org/archive/html/emacs-devel/2016-07/msg00852.html

In the previous discussion someone said that we would be punishing
users that do not use it if we preloaded it.  It's not clear to me how
this would punish users.  On the contrary, if it was preloaded, more
powerful abstractions would make it easier to maintain Emacs and free
time for developers by not forcing us to reinvent wheels.  This would
improve Emacs and help all users.

Another argument was that seq.el is not used that much.  This is not
surprising, since it's not preloaded.  It's therefore hard to use in
some places of Emacs.  If it was preloaded, it would presumably see
more use.

Stefan Monnier also pointed out that seq.el depends on cl-lib, and
that to preload seq, we would need to preload cl-lib.  This seems to
be due to a single use of the  cl-subseq function in cl-extra.el.  If
this is a show stopper, perhaps we could see if we can get rid of that
single use.

Best regards,
Stefan Kangas

Re: Preloading seq.el

[Lars Ingebrigtsen]

Stefan Kangas <stefan@marxist.se> writes:

> Stefan Monnier also pointed out that seq.el depends on cl-lib, and
> that to preload seq, we would need to preload cl-lib.  This seems to
> be due to a single use of the  cl-subseq function in cl-extra.el.  If
> this is a show stopper, perhaps we could see if we can get rid of that
> single use.

I'm all for preloading both seq.el and cl-lib.el.  I think it's annoying
that we're not allowed to use these very useful libraries in the most
central bits of Emacs, but instead have to write less optimal and
readable code in those files.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Preloading seq.el

[Eli Zaretskii]

> From: Stefan Kangas <stefan@marxist.se>
> Date: Thu, 29 Aug 2019 14:21:00 +0200
> 
> Could we preload seq.el?
> 
> Previous discussion here:
> https://lists.gnu.org/archive/html/emacs-devel/2016-07/msg00852.html
> 
> In the previous discussion someone said that we would be punishing
> users that do not use it if we preloaded it.  It's not clear to me how
> this would punish users.

By enlarging the memory footprint of a running Emacs more than
necessary.

> Stefan Monnier also pointed out that seq.el depends on cl-lib, and
> that to preload seq, we would need to preload cl-lib.  This seems to
> be due to a single use of the  cl-subseq function in cl-extra.el.  If
> this is a show stopper, perhaps we could see if we can get rid of that
> single use.

I don't yet understand what problems do you have with having the
necessary functions from seq.el autoloaded.  Is the code you are
writing for help.el meant to run in every Emacs session right at its
beginning?  If not, why autoloading is not the solution?

Re: Preloading seq.el

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Thu, 29 Aug 2019 14:30:57 +0200
> Cc: emacs-devel@gnu.org
> 
> I'm all for preloading both seq.el and cl-lib.el.  I think it's annoying
> that we're not allowed to use these very useful libraries in the most
> central bits of Emacs, but instead have to write less optimal and
> readable code in those files.

Preloading is a far cry from being allowed to use the packages.  I
don't understand why you consider not preloading a package as if that
denies your ability to use it.

Re: Preloading seq.el

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> I'm all for preloading both seq.el and cl-lib.el.  I think it's annoying
>> that we're not allowed to use these very useful libraries in the most
>> central bits of Emacs, but instead have to write less optimal and
>> readable code in those files.
>
> Preloading is a far cry from being allowed to use the packages.  I
> don't understand why you consider not preloading a package as if that
> denies your ability to use it.

I was talking about using seq/cl-lib in the parts of Emacs that's dumped
(like help.el), which is currently not allowed, because that would
require preloading seq/cl-lib.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Preloading seq.el

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: stefan@marxist.se,  emacs-devel@gnu.org
> Date: Thu, 29 Aug 2019 14:50:10 +0200
> 
> I was talking about using seq/cl-lib in the parts of Emacs that's dumped
> (like help.el), which is currently not allowed, because that would
> require preloading seq/cl-lib.

I'm probably missing something, because I don't understand how using
SOMETHING.el from a preloaded code would necessarily requore
preloading SOMETHING.el as well.  Isn't autoloading working?

Re: Preloading seq.el

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> I was talking about using seq/cl-lib in the parts of Emacs that's dumped
>> (like help.el), which is currently not allowed, because that would
>> require preloading seq/cl-lib.
>
> I'm probably missing something, because I don't understand how using
> SOMETHING.el from a preloaded code would necessarily requore
> preloading SOMETHING.el as well.  Isn't autoloading working?

Oh, I didn't think slapping ;;;###autoload cookies on all the functions
in seq/cl-lib was on the table, but if it is, that's a solution, too.

By the way, I summed up the size of the .elc files loadup.el today does,
and it's over 3MB.  seq/cl-lib.elc is 56KB.  So we're talking about
preloading less than 2% more .elc code.  And there's 242 .el files that
require seq/cl-lib, so I think it's unlikely that many users don't end
up with loading seq/cl-lib, anyway, no matter what they do beyond
starting "emacs -Q".

(And about the same number that does `eval-when-require' on those, but
that's a different story...)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Preloading seq.el

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: stefan@marxist.se,  emacs-devel@gnu.org
> Date: Thu, 29 Aug 2019 17:15:27 +0200
> 
> > I'm probably missing something, because I don't understand how using
> > SOMETHING.el from a preloaded code would necessarily requore
> > preloading SOMETHING.el as well.  Isn't autoloading working?
> 
> Oh, I didn't think slapping ;;;###autoload cookies on all the functions
> in seq/cl-lib was on the table, but if it is, that's a solution, too.

I don't know if we want all of them to be autoloaded, but I see no
reason not to autoload some, those which are used by preloaded code.

> By the way, I summed up the size of the .elc files loadup.el today does,
> and it's over 3MB.  seq/cl-lib.elc is 56KB.  So we're talking about
> preloading less than 2% more .elc code.  And there's 242 .el files that
> require seq/cl-lib, so I think it's unlikely that many users don't end
> up with loading seq/cl-lib, anyway, no matter what they do beyond
> starting "emacs -Q".

The way to hell is paced one step at a time.  Those 3MB were not all
added in one go, they were added one file at a time.  So we need to
examine every single file we want to add and decide on a case by case
basis, otherwise we will get 6MB instead of 3MB faster than you can
say "make autoloads".

Re: Preloading seq.el

[Stefan Monnier]

>> Stefan Monnier also pointed out that seq.el depends on cl-lib, and
>> that to preload seq, we would need to preload cl-lib.

That's only a technical detail, tho: If we decide to preload seq.el it
should be easy to reverse the dependency (i.e. make cl-lib use seq.el
rather than the reverse).

> I'm all for preloading both seq.el and cl-lib.el.

I think I'm mildly favorable as well, but I'm not very interested in
putting the work needed for this to happen (IIRC there are some
technical issues to address before we can do that).

> I think it's annoying that we're not allowed to use these very useful
> libraries in the most central bits of Emacs, but instead have to write
> less optimal and readable code in those files.

You can see my addition of pcase, lexical-binding, setf, cl-preloaded,
and cl-generic as a way to reduce this tension ;-)


        Stefan

Re: Preloading seq.el

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> The way to hell is paced one step at a time.  Those 3MB were not all
> added in one go, they were added one file at a time.  So we need to
> examine every single file we want to add and decide on a case by case
> basis, otherwise we will get 6MB instead of 3MB faster than you can
> say "make autoloads".

Indeed.  But in my experience, I find myself using seq/cl-lib functions
in the pre-loaded Emacs files (because I just don't have a very good
overview of which ones are preloaded and which ones aren't), and then
having to rewrite the code to avoid using those functions when it turns
out that Emacs doesn't build.

(Sometimes after pushing, unfortunately.)

So I'm saying that, in my opinion, seq/cl-lib would be good additions to
the set of preloaded packages in Emacs.  They would make the code in
those files better, and may well make them smaller in the long run
because we wouldn't have to basically open-code the seq/cl-lib
functionality, which I find myself doing now.

(But if you're OK with ;;;###autoload any time we use seq/cl-lib in
those files, then that's also fine with me, but I think that in the long
run, it's preferable to just preload them, because I think we're going
to end up with 80% of them ;;;###autoloaded.)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Preloading seq.el

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: stefan@marxist.se,  emacs-devel@gnu.org
> Date: Thu, 29 Aug 2019 17:35:06 +0200
> 
> (But if you're OK with ;;;###autoload any time we use seq/cl-lib in
> those files, then that's also fine with me, but I think that in the long
> run, it's preferable to just preload them, because I think we're going
> to end up with 80% of them ;;;###autoloaded.)

When we come to the point that 80% of seq.el is used in preloaded
code, let's talk again.

And please note that it is also important _when_ those functions are
called and under what circumstances.  For example, descr-text.el is
autoloaded whenever one types "C-u C-x =", which invokes a function
defined in the preloaded simple.el, but we don't preload descr-text.el
even though almost all of it is needed when that command is invoked.
The reason is that "C-u C-x =" is not invoked frequently enough to
justify preloading descr-text.el.  So we need to consider not only the
amount of called code, but also its callers.

Re: Preloading seq.el

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> (But if you're OK with ;;;###autoload any time we use seq/cl-lib in
>> those files, then that's also fine with me, but I think that in the long
>> run, it's preferable to just preload them, because I think we're going
>> to end up with 80% of them ;;;###autoloaded.)
>
> When we come to the point that 80% of seq.el is used in preloaded
> code, let's talk again.

That's fine with me, and I guess we should now start adding
;;;###autoload to seq.el whenever we feel like using functions from that
file from the preloaded files.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Preloading seq.el

[Stefan Monnier]

> That's fine with me, and I guess we should now start adding
> ;;;###autoload to seq.el whenever we feel like using functions from that
> file from the preloaded files.

[ Assuming this option does, which is not a given.  ]

Yes, I think it's a good first step: it will also *document* the places
where seq/cl-lib is needed so we can make a more informed choice.

When it doesn't work, adding a comment showing the code we'd use if we
could use seq/cl-lib could play a similar role.


        Stefan

Re: Preloading seq.el

[Zhu Zihao]

I also prefer preload cl-lib because it's widely used in Emacs builtin package even whole
emacsen. 

There's 377 references of "(require 'cl-lib)" in "lisp" folder of emacs source
repository, but only 18 references to "(require 'seq)" 

Re: Preloading seq.el

[Eli Zaretskii]

> Date: Sat, 31 Aug 2019 20:33:20 +0800
> From: Zhu Zihao <all_but_last@163.com>
> Cc: emacs-devel@gnu.org
> 
> I also prefer preload cl-lib because it's widely used in Emacs builtin package even whole
> emacsen. 
> 
> There's 377 references of "(require 'cl-lib)" in "lisp" folder of emacs source
> repository, but only 18 references to "(require 'seq)" 

Once again, the number of packages that require cl-lib is not an
important factor to consider when deciding whether to preload it.
Preloading a package has almost nothing to do with requiring it from
other packages, it has top do with its use in a fresh "emacs -Q".

Re: Preloading seq.el

[Juanma Barranquero]

[-- Attachment #1: Type: text/plain, Size: 693 bytes --]

> Once again, the number of packages that require cl-lib is not an
> important factor to consider when deciding whether to preload it.

A better metrics (but hard to estimate) is how many times a function of
cl-lib has been reimplemented (likely in a much simpler way), or how many
workarounds are in place because someone wanted to use a cl-lib function in
the pre-loaded code, but couldn't, and didn't have the energy to try to
convince emacs-devel to preload cl-lib ;-)

I mean, cl-lib is infrastructure code. It's useful and expressive. If it
were preloaded, people would find uses for it. Whether that is a good
argument for inclusion or not... Yes, to me. But obviously YMMV.

   Juanma

[-- Attachment #2: Type: text/html, Size: 804 bytes --]

Re: Preloading seq.el

[Eli Zaretskii]

> From: Juanma Barranquero <lekktu@gmail.com>
> Date: Fri, 6 Sep 2019 02:49:24 +0200
> Cc: Zhu Zihao <all_but_last@163.com>, stefan@marxist.se, 
> 	Emacs developers <emacs-devel@gnu.org>
> 
> A better metrics (but hard to estimate) is how many times a function of cl-lib has been reimplemented (likely in
> a much simpler way), or how many workarounds are in place because someone wanted to use a cl-lib
> function in the pre-loaded code, but couldn't, and didn't have the energy to try to convince emacs-devel to
> preload cl-lib ;-)

Only those of the cases where autoloading wouldn't have solved the
issue.

> I mean, cl-lib is infrastructure code. It's useful and
> expressive. If it were preloaded, people would find uses for it.

They can use it now by autoloading cl-lib functions they need.

Re: Preloading seq.el

[Robert Pluim]

>>>>> On Fri, 06 Sep 2019 10:12:54 +0300, Eli Zaretskii <eliz@gnu.org> said:

    >> From: Juanma Barranquero <lekktu@gmail.com>
    >> Date: Fri, 6 Sep 2019 02:49:24 +0200
    >> Cc: Zhu Zihao <all_but_last@163.com>, stefan@marxist.se, 
    >> Emacs developers <emacs-devel@gnu.org>
    >> 
    >> A better metrics (but hard to estimate) is how many times a function of cl-lib has been reimplemented (likely in
    >> a much simpler way), or how many workarounds are in place because someone wanted to use a cl-lib
    >> function in the pre-loaded code, but couldn't, and didn't have the energy to try to convince emacs-devel to
    >> preload cl-lib ;-)

    Eli> Only those of the cases where autoloading wouldn't have solved the
    Eli> issue.

    >> I mean, cl-lib is infrastructure code. It's useful and
    >> expressive. If it were preloaded, people would find uses for it.

    Eli> They can use it now by autoloading cl-lib functions they need.

Hmm, if cl-lib is that useful, could we not make people's lives easier
by just slapping autoload cookies on all of it?

Robert

Re: Preloading seq.el

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Cc: Juanma Barranquero <lekktu@gmail.com>,  stefan@marxist.se,
>   all_but_last@163.com,  emacs-devel@gnu.org
> Date: Fri, 06 Sep 2019 09:46:28 +0200
> 
>     Eli> They can use it now by autoloading cl-lib functions they need.
> 
> Hmm, if cl-lib is that useful, could we not make people's lives easier
> by just slapping autoload cookies on all of it?

We could, if it turns out a large percentage ends up autoloaded.  But
I'd like first to see whether this is the case, by watching the growth
of autoload cookies in cl-lib over time.  We don't lose anything by
doing this incrementally rather than in bulk, do we?

Re: Preloading seq.el

[Juanma Barranquero]

[-- Attachment #1: Type: text/plain, Size: 161 bytes --]

On Fri, Sep 6, 2019 at 9:13 AM Eli Zaretskii <eliz@gnu.org> wrote:

> Only those of the cases where autoloading wouldn't have solved the
> issue.

You're right.

[-- Attachment #2: Type: text/html, Size: 293 bytes --]