bug#43678: 27.1; compiler warning if code forgets to require cl-lib

bug#43678: 27.1; compiler warning if code forgets to require cl-lib

[Roland Winkler]

Until earlier today, several files in bbdb did not require cl-lib,
when they should have done that.  Nonetheless, bbdb compiled fine
with emacs 25 and 26, but not anymore with emacs 27, see bug #30635.
The goal when fixing #30635 was to issue a warning if a library uses
cl-lib without requiring it.  The problem with bbdb was, however,
that instead the emacs 27 byte compiler threw a rather unhelpful
error message about the cause of the problem.  For bbdb-tex.el, it
said

In bbdb-TeX:
bbdb-tex.el:414:25:Error: Forgot to expand macro cl-progv in ( ...

I suggest that the byte compiler should issue a more helpful warning
about the cause of the problem.



In GNU Emacs 27.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.18.9)
 of 2020-08-31 built on regnitz
Windowing system distributor 'The X.Org Foundation', version 11.0.11804000
System Description: Ubuntu 16.04.7 LTS

bug#43678: 27.1; compiler warning if code forgets to require cl-lib

[Stefan Monnier]

> Until earlier today, several files in bbdb did not require cl-lib,
> when they should have done that.  Nonetheless, bbdb compiled fine
> with emacs 25 and 26, but not anymore with emacs 27, see bug #30635.
> The goal when fixing #30635 was to issue a warning if a library uses
> cl-lib without requiring it.  The problem with bbdb was, however,
> that instead the emacs 27 byte compiler threw a rather unhelpful
> error message about the cause of the problem.  For bbdb-tex.el, it
> said
>
> In bbdb-TeX:
> bbdb-tex.el:414:25:Error: Forgot to expand macro cl-progv in ( ...
>
> I suggest that the byte compiler should issue a more helpful warning
> about the cause of the problem.

Could you show how you got that error?
When I try to compile bbdb-tex.el I get the following:

    % src/emacs -L .../elpa/packages/bbdb/ -Q --batch -f batch-byte-compile .../elpa/packages/bbdb/bbdb-tex.el
    
    In end of data:
    .../elpa/packages/bbdb/bbdb-tex.el:568:1: Warning: the function ‘cl-progv’ is
        not known to be defined.
    %

Which is the source of the error and which I hoped was "helpful enough".

This is just a "warning" but it actually results in the file being
miscompiled.  The byte-compiler only issues a warning rather than an
error because it doesn't know `cl-progv` (and hence doesn't know that
it's a macro either) at that point and hence just assumes that it will
be defined as a function.

If you later load that `bbdb-tex.elc` file and use the `bbdb-tex`
function, when it gets to the point of executing the code with the
`cl-progv` it will indeed signal something like the error you've shown
(because the byte code tries to call the `cl-progv` *function* but finds
a *macro* in its stead).


        Stefan

bug#43678: 27.1; compiler warning if code forgets to require cl-lib

[Roland Winkler]

On Mon Sep 28 2020 Stefan Monnier wrote:
> Could you show how you got that error?

I remove the line

  (eval-when-compile (require 'cl-lib))

from bbdb-tex.el, then I run

  $ /path_to_emacs_27.1/emacs --batch --directory=./  --funcall batch-byte-compile bbdb-tex.el

This gives me

In bbdb-tex:
bbdb-tex.el:415:25:Error: Forgot to expand macro cl-progv in (cl-progv ...

I was not the only one with this problem, see

https://lists.nongnu.org/archive/html/bbdb-user/2020-09/msg00000.html

bug#43678: 27.1; compiler warning if code forgets to require cl-lib

[Stefan Monnier]

> In bbdb-tex:
> bbdb-tex.el:415:25:Error: Forgot to expand macro cl-progv in (cl-progv ...

Oh, yes, now I can reproduce it as well (I was using an older version
of BBDB before).  And indeed, I had misunderstood the error, this one is
a different one from the one I was thinking of.

This error is because `cl-progv` was not known to the compiler when we
started compiling the `bbdb-tex` function, but during the course of its
compilation, something somewhere caused `cl-lib` to be loaded such that
by the time the byte-compiler gets to look again at (cl-progv ...), it
now notices that it's a macro, which should not be possible because we
should have performed all the macro-expansion at the very
beginning already.

Hmmm... indeed it's clearly not a very helpful message since it even
misled its original author.  I think when I wrote it I assumed it would
only occur because of an internal bug, but clearly it can occur in
other cases.

I'm not sure what would be a good error message.
Maybe something like "`cl-progv` defined too late"?


        Stefan

bug#43678: 27.1; compiler warning if code forgets to require cl-lib

[Roland Winkler]

On Mon Sep 28 2020 Stefan Monnier wrote:
> I'm not sure what would be a good error message.
> Maybe something like "`cl-progv` defined too late"?

This is a tough question in the sense that presumably this error
message should never appear in the first place.  Could it make sense
that the error message mentioned something like "cl-lib may have
been loaded too late"?  What other scenarios besides not requiring
cl-lib can possibly be the ultimate cause of this error message so
that this message can give the user some hint how to solve this
problem?

bug#43678: 27.1; compiler warning if code forgets to require cl-lib

[Stefan Monnier]

>> I'm not sure what would be a good error message.
>> Maybe something like "`cl-progv` defined too late"?
> This is a tough question in the sense that presumably this error
> message should never appear in the first place.  Could it make sense
> that the error message mentioned something like "cl-lib may have
> been loaded too late"?  What other scenarios besides not requiring
> cl-lib can possibly be the ultimate cause of this error message so
> that this message can give the user some hint how to solve this
> problem?

This is not specific to `cl-lib`.  It can happen with any macro if you
forget to `require` the file that provides this macro, but the macro
expansion (including compiler macros's expansion, i.e. optimizations) of
some of the rest of your function ends up loading the file that provides
this macro.


        Stefan

bug#43678: 27.1; compiler warning if code forgets to require cl-lib

[Roland Winkler]

On Mon Sep 28 2020 Stefan Monnier wrote:
> This is not specific to `cl-lib`.  It can happen with any macro if you
> forget to `require` the file that provides this macro, but the macro
> expansion (including compiler macros's expansion, i.e. optimizations) of
> some of the rest of your function ends up loading the file that provides
> this macro.

Then it seems to me that the error message could say that possibly we
forgot to require *a* library.   (You surely know the speculations
about the cause of a problem issued by the TeX compiler.  I do not
remember I ever saw such speculations coming from Emacs.  Maybe the
byte compiler should first run a session with a doctor
(M-x emacs-psychotherapist) to narrow down the cause of the
problem...)

bug#43678: 27.1; compiler warning if code forgets to require cl-lib

[Lars Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> I'm not sure what would be a good error message.
> Maybe something like "`cl-progv` defined too late"?

Or perhaps

"`cl-progv` defined too late (missing `require' of a library file?)"

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#43678: 27.1; compiler warning if code forgets to require cl-lib

[Howard Melman]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
>> I'm not sure what would be a good error message.
>> Maybe something like "`cl-progv` defined too late"?
>
> Or perhaps
>
> "`cl-progv` defined too late (missing `require' of a library file?)"

I think I'd find "after use" clearer than "too late".

-- 

Howard

bug#43678: 27.1; compiler warning if code forgets to require cl-lib

[Lars Ingebrigtsen]

Howard Melman <hmelman@gmail.com> writes:

>> "`cl-progv` defined too late (missing `require' of a library file?)"
>
> I think I'd find "after use" clearer than "too late".

Good idea.  I've now pushed this version to Emacs 28.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no