bug#70108: 29.1.90; `defalias' and `current-load-list'

bug#70108: 29.1.90; `defalias' and `current-load-list'

[Mickey Petersen]

Here's a curious one. This is not so much a bug, I think, as a gotcha
and perhaps something that I wouldn't mind understanding a bit better.

My package creates a number of dynamic commands (think:
tempo/skeletons) to expand text.

It does so using the common (defalias NAME (lambda () ...)) pattern.

This obviously goes into the C core where a bunch of stuff takes
place. One of variables that gets updated (somehow) is
`current-load-list'. Curiously, it ends up with duplicate entries. I'm
presuming there is a good reason for this.

This variable can easily get overrun with identical entries if the
inattentive programmer does not check if it is already bound.

Due to a screwup on my part, I did not `fboundp' check first, which had the
odd effect of causing incredible slowdowns when I ran my 600+ ERT test
suite.

I have not narrowed down exactly *why* my ERT suite, when beset by 600
tests to run in one go, causes the slowdown. What I'm guessing from
how each successive tests slows down, that there is some sort of
non-linear searching going on here, and either `load' (and friends) or
the mere fact that putting more defaliases into the system causes this
slowdown to occur.

Anyway. Not sure it's a bug; but it caught me out, and as
`current-load-list' is rather obscure, I figured I'd at least ask.

Thanks!

Mickey.

bug#70108: 29.1.90; `defalias' and `current-load-list'

[Eli Zaretskii]

> From: Mickey Petersen <mickey@masteringemacs.org>
> Date: Sun, 31 Mar 2024 16:34:37 +0100
> 
> Here's a curious one. This is not so much a bug, I think, as a gotcha
> and perhaps something that I wouldn't mind understanding a bit better.
> 
> My package creates a number of dynamic commands (think:
> tempo/skeletons) to expand text.
> 
> It does so using the common (defalias NAME (lambda () ...)) pattern.
> 
> This obviously goes into the C core where a bunch of stuff takes
> place. One of variables that gets updated (somehow) is
> `current-load-list'. Curiously, it ends up with duplicate entries. I'm
> presuming there is a good reason for this.
> 
> This variable can easily get overrun with identical entries if the
> inattentive programmer does not check if it is already bound.
> 
> Due to a screwup on my part, I did not `fboundp' check first, which had the
> odd effect of causing incredible slowdowns when I ran my 600+ ERT test
> suite.
> 
> I have not narrowed down exactly *why* my ERT suite, when beset by 600
> tests to run in one go, causes the slowdown. What I'm guessing from
> how each successive tests slows down, that there is some sort of
> non-linear searching going on here, and either `load' (and friends) or
> the mere fact that putting more defaliases into the system causes this
> slowdown to occur.

Adding Stefan.

bug#70108: 29.1.90; `defalias' and `current-load-list'

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> This obviously goes into the C core where a bunch of stuff takes
>> place. One of variables that gets updated (somehow) is
>> `current-load-list'. Curiously, it ends up with duplicate entries. I'm
>> presuming there is a good reason for this.

Duplicate entries sounds like a bug (unless there are two calls to
`defalias` for the same function name within the same file).
Do you have a reproducible recipe that shows this?

>> This variable can easily get overrun with identical entries if the
>> inattentive programmer does not check if it is already bound.

Hmmm... I'm beginning to wonder: do your `defalias` happen while loading
a file, or do they happen more "dynamically"?

>> I have not narrowed down exactly *why* my ERT suite, when beset by 600
>> tests to run in one go, causes the slowdown. What I'm guessing from
>> how each successive tests slows down, that there is some sort of
>> non-linear searching going on here, and either `load' (and friends) or
>> the mere fact that putting more defaliases into the system causes this
>> slowdown to occur.

The most common culprit is "N linear searches in an N-long list".


        Stefan

bug#70108: 29.1.90; `defalias' and `current-load-list'

[Mickey Petersen]

Hi Stefan,


Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>> This obviously goes into the C core where a bunch of stuff takes
>>> place. One of variables that gets updated (somehow) is
>>> `current-load-list'. Curiously, it ends up with duplicate entries. I'm
>>> presuming there is a good reason for this.
>
> Duplicate entries sounds like a bug (unless there are two calls to
> `defalias` for the same function name within the same file).
> Do you have a reproducible recipe that shows this?
>

That is exactly the scenario. There's a setup function that gets
called, which in turn sets up the defaliases. Each test would then end
up calling this setup function, which would result in the same file
foo.el calling defalias multiple times for things it has already
defined.

Still, I find it a bit odd that it's *that* slow. We're talking
hundreds, not millions, of entries.

As I mentioned earlier, I do not think it's a bug, but it caught me
out, so I figured I'd at least ask.

>>> This variable can easily get overrun with identical entries if the
>>> inattentive programmer does not check if it is already bound.
>
> Hmmm... I'm beginning to wonder: do your `defalias` happen while loading
> a file, or do they happen more "dynamically"?
>

The file is loaded once as part of the batch emacs process to run the
tests, and unless there's some ert shenanigans I'm not familiar with,
I do not think so.

>>> I have not narrowed down exactly *why* my ERT suite, when beset by 600
>>> tests to run in one go, causes the slowdown. What I'm guessing from
>>> how each successive tests slows down, that there is some sort of
>>> non-linear searching going on here, and either `load' (and friends) or
>>> the mere fact that putting more defaliases into the system causes this
>>> slowdown to occur.
>
> The most common culprit is "N linear searches in an N-long list".
>         Stefan

bug#70108: 29.1.90; `defalias' and `current-load-list'

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> Duplicate entries sounds like a bug (unless there are two calls to
>> `defalias` for the same function name within the same file).
>> Do you have a reproducible recipe that shows this?
> That is exactly the scenario.  There's a setup function that gets
> called, which in turn sets up the defaliases.  Each test would then end
> up calling this setup function, which would result in the same file
> foo.el calling defalias multiple times for things it has already
> defined.

Ah, OK, so it sounds like a misfeature in your code which should
presumably avoid doing the setup repeatedly.
[ As opposed to the case where those "duplicate" definitions end up
  defining the functions differently.  ]

Good.  That means that you can easily work around the current
performance problem.

> Still, I find it a bit odd that it's *that* slow.

How slow are we talking about?
Have you tried `M-x profiler-start RET RET` and then later
`M-x profiler-report RET`?

In the absence of a reproducible recipe, I'd be interested to see where
the time is spent.

> We're talking hundreds, not millions, of entries.

You mentioned 600, so if there's the usual O(N²) behavior, that quickly
gets us to 100K territory which can be slow enough to be noticeable.

> As I mentioned earlier, I do not think it's a bug, but it caught me
> out, so I figured I'd at least ask.

It does sound like a performance bug, tho (triggered by an undesirable
behavior in your code).
We fixed a similar problem in `defvar` in the past.

>>>> This variable can easily get overrun with identical entries if the
>>>> inattentive programmer does not check if it is already bound.
>> Hmmm... I'm beginning to wonder: do your `defalias` happen while loading
>> a file, or do they happen more "dynamically"?
> The file is loaded once as part of the batch emacs process to run the
> tests, and unless there's some ert shenanigans I'm not familiar with,
> I do not think so.

But are the problematic `defalias` performed while loading the file
that defines the tests, or doing the execution of those tests.

That affects for example whether those function definitions end up
appearing in `load-history` as belonging to the file or belonging to
"unknown".


        Stefan