byte compiler warnings when bootstrapping

byte compiler warnings when bootstrapping

[Dan Nicolaescu]

When bootstrapping the number of byte compiler warnings is huge.

A few bugs could have been caught earlier if the problematic warnings
would have been easier to find.

Wouldn't it be better if an effort would be made to get rid of most
(if not all) the warnings?

To get rid of some of the "function is not known to be defined"
warnings it would be good if we had a way to tell the byte compiler
to not warn about a function not being defined. 

With that and with adding a few more (eval-when-compile (require 'blah))
we should be able to get rid of most warnings. 

Is this a worthy goal? Opinions? 

Re: byte compiler warnings when bootstrapping

[martin rudalics]

> When bootstrapping the number of byte compiler warnings is huge.
> 
> A few bugs could have been caught earlier if the problematic warnings
> would have been easier to find.
> 
> Wouldn't it be better if an effort would be made to get rid of most
> (if not all) the warnings?
> 
> To get rid of some of the "function is not known to be defined"
> warnings it would be good if we had a way to tell the byte compiler
> to not warn about a function not being defined. 
> 
> With that and with adding a few more (eval-when-compile (require 'blah))
> we should be able to get rid of most warnings. 
> 
> Is this a worthy goal? Opinions? 

Very worthy, IMHO.

Thanks also for taking care of the next-/previous-line annoyances.

Re: byte compiler warnings when bootstrapping

[Stefan Monnier]

> With that and with adding a few more (eval-when-compile (require 'blah))

What is the use of (eval-when-compile (require 'blah)) in this context?
I expect it's the wrong solution.


        Stefan

Re: byte compiler warnings when bootstrapping

[Dan Nicolaescu]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

  > > With that and with adding a few more (eval-when-compile (require 'blah))
  > 
  > What is the use of (eval-when-compile (require 'blah)) in this context?
  > I expect it's the wrong solution.

When the number of warnings for undefined functions is too high it
might be better to just require the package that defines them. But you
might be right...

Re: byte compiler warnings when bootstrapping

[David Kastrup]

Dan Nicolaescu <dann@ics.uci.edu> writes:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
>   > > With that and with adding a few more (eval-when-compile (require 'blah))
>   > 
>   > What is the use of (eval-when-compile (require 'blah)) in this context?
>   > I expect it's the wrong solution.
>
> When the number of warnings for undefined functions is too high it
> might be better to just require the package that defines them. But you
> might be right...

But how do the functions actually get defined at runtime?

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: byte compiler warnings when bootstrapping

[Dan Nicolaescu]

David Kastrup <dak@gnu.org> writes:

  > Dan Nicolaescu <dann@ics.uci.edu> writes:
  > 
  > > Stefan Monnier <monnier@iro.umontreal.ca> writes:
  > >
  > >   > > With that and with adding a few more (eval-when-compile (require 'blah))
  > >   > 
  > >   > What is the use of (eval-when-compile (require 'blah)) in this context?
  > >   > I expect it's the wrong solution.
  > >
  > > When the number of warnings for undefined functions is too high it
  > > might be better to just require the package that defines them. But you
  > > might be right...
  > 
  > But how do the functions actually get defined at runtime?

Autoloading

(defun foo ()
  (info)
  (Info-read-node-name) <- this gets a byte compiler a warning,
                           but it is defined at runtime.

Re: byte compiler warnings when bootstrapping

[David Kastrup]

Dan Nicolaescu <dann@ics.uci.edu> writes:

> David Kastrup <dak@gnu.org> writes:
>
>   > Dan Nicolaescu <dann@ics.uci.edu> writes:
>   > 
>   > > Stefan Monnier <monnier@iro.umontreal.ca> writes:
>   > >
>   > >   > > With that and with adding a few more (eval-when-compile
>   > >   > > (require 'blah))
>   > >   > 
>   > >   > What is the use of (eval-when-compile (require 'blah)) in
>   > >   > this context?  I expect it's the wrong solution.
>   > >
>   > > When the number of warnings for undefined functions is too high it
>   > > might be better to just require the package that defines them. But you
>   > > might be right...
>   > 
>   > But how do the functions actually get defined at runtime?
>
> Autoloading
>
> (defun foo ()
>   (info)
>   (Info-read-node-name) <- this gets a byte compiler a warning,

That is not a good idea since it relies on info and
Info-read-node-name being loaded from the same file.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: byte compiler warnings when bootstrapping

[Stefan Monnier]

>> > With that and with adding a few more (eval-when-compile (require 'blah))
>> What is the use of (eval-when-compile (require 'blah)) in this context?
>> I expect it's the wrong solution.
> When the number of warnings for undefined functions is too high it
> might be better to just require the package that defines them.  But you
> might be right...

The bytecomp.el has code (which ssems not to work any more) that tries to
re-add the warnings in this case (turns the "unknown function" warnings you
get without the `require' into "not sure it'll be defined at runtime" with
the eval-when-compile require).

So it's not the right approach.  `eval-when-compile' should really be used
to require packages that provide a macro or something like that: something
you really need to properly compile the file.

I agree that we need something to turn off specific warnings.  Someone at
some point suggested (defun <foo>) following the same idea as (defvar <foo>).

I think it would be good.  Although I think I'd rather allow the
external-function-delcaration to give an arglist so that the bytecompiler
can check you passed the right number of arguments as well.  So maybe
something like

  (declare-function <foo> <fooarglist>)

would work better.

Along similar lines I'd like to add a

  (require-lazily <feature>)

which would behave similarly to `require' except that the byte-compiler
would replace it with a bunch of autoloads.  so the package would only be
"required" when one of its functions was used.

In any case, for the issue of "dependencies" (e.g. calling
`Info-read-node-name' after `info', or calling `help-buffer' after
`help-setup-xref') I'm not sure what we should ideally do, but the
require-lazily seems like an acceptable solution.


        Stefan

Re: byte compiler warnings when bootstrapping

[Bob Rogers]

   From: Stefan Monnier <monnier@iro.umontreal.ca>
   Date: Sun, 21 Oct 2007 15:38:06 -0400

   . . .

   Along similar lines I'd like to add a

     (require-lazily <feature>)

   which would behave similarly to `require' except that the byte-compiler
   would replace it with a bunch of autoloads.  so the package would only be
   "required" when one of its functions was used.

   In any case, for the issue of "dependencies" (e.g. calling
   `Info-read-node-name' after `info', or calling `help-buffer' after
   `help-setup-xref') I'm not sure what we should ideally do, but the
   require-lazily seems like an acceptable solution.

	   Stefan

Since (IIUC) these are "internal" functions in one package that are
called from another, it seems better to mark them as such at the point
of definition, rather than in some random place in the files that call
them.

   For example, suppose we defined a ";;;API" tag line (or preferably
something less cryptic) to put before the definitions of functions that
are called elsewhere, by analogy with ";;;AUTOLOAD".  Then, a function
like batch-update-autoloads could be written that extracted these tagged
definitions into "declare-function" forms, building an "api-defs.el"
file that would be loaded into batch-byte-compile jobs in order to
suppress the "undefined function" warnings without requiring them to be
loaded.  This also has the benefit of explicitly identifying function
definitions that are used elsewhere within Emacs, despite seeming to be
internal, as a warning to people who might otherwise carelessly change
their interface.

   Given that much, it then ought to be trivial for the byte compiler to
insert

	(or (fbound 'foo-api-function)
	    (require 'foo))

(or something similar to what autoloading does) before the point of call
for each such function, giving the same effect as "require-lazily".

   Note that "api-defs.el" would not be loaded into emacs except when
compiling, nor would it ever be installed.  That way, these "internal
API" functions would remain internal as far as non-Emacs packages are
concerned.

   Thoughts?

					-- Bob Rogers
					   http://rgrjr.dyndns.org/

Re: byte compiler warnings when bootstrapping

[Richard Stallman]

I would like to do something to silence the manu "function is not
known to be defined" warnings that don't correspond to anything wrong.  But
loading another file just for that is too drastic for the purpose.

Perhaps we should add a construct

  (treat-as-defined FILE foo bar)

which would tell the compiler not to warn in this file
about calls to foo and bar -- and have no other effect.

Some special program could scan the sources looking for all these
statements, and then verify that FILE really does define foo and bar.
This could be run during the build process, but not during ordinary
byte compilation.

Re: byte compiler warnings when bootstrapping

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Date: Mon, 22 Oct 2007 05:00:18 -0400
> Cc: dann@ics.uci.edu, emacs-devel@gnu.org
> 
> Perhaps we should add a construct
> 
>   (treat-as-defined FILE foo bar)
> 
> which would tell the compiler not to warn in this file
> about calls to foo and bar -- and have no other effect.

The name ``treat-as-defined'' is not telling enough what is its
effect.  How about a more general facility, like this:

  (bytecomp-no-warn 'undefined-function FILE FUNC1 FUNC2 ...)

Re: byte compiler warnings when bootstrapping

[Richard Stallman]

    I think it would be good.  Although I think I'd rather allow the
    external-function-delcaration to give an arglist so that the bytecompiler
    can check you passed the right number of arguments as well.  So maybe
    something like

      (declare-function <foo> <fooarglist>)

That seems like a good design, but I think it should also have
the file name, so that something at build time can check all these
against the files they refer to.

    Along similar lines I'd like to add a

      (require-lazily <feature>)

    which would behave similarly to `require' except that the byte-compiler
    would replace it with a bunch of autoloads.  so the package would only be
    "required" when one of its functions was used.

I have a bad feeling about that -- that it would produce scads of autoloads
which are not needed.  Also, I don't think it is needed in any of the existing
cases of warnings we would like to suppress.

Re: byte compiler warnings when bootstrapping

[Richard Stallman]

       For example, suppose we defined a ";;;API" tag line (or preferably
    something less cryptic) to put before the definitions of functions that
    are called elsewhere, by analogy with ";;;AUTOLOAD".  Then, a function
    like batch-update-autoloads could be written that extracted these tagged
    definitions into "declare-function" forms, building an "api-defs.el"
    file that would be loaded into batch-byte-compile jobs in order to
    suppress the "undefined function" warnings without requiring them to be
    loaded.

This approach could, but putting them in one large file "api-defs.el"
would only do the job if we also make in each file state which other
files it really wants to depend on.  You'd need something like
(compile-require 'foo) to tell the compiler to consider the api-defs
info for foo.

Also, this would only work for calling functions in files
that were scanned by the batch process.  And if you update
those files, you would have to rerun the batch process.

Thus, I think my proposal is better in several ways.
It would be just as reliable, for calls to files checked
by the batch process, but it would not limit you.

Re: byte compiler warnings when bootstrapping

[Richard Stallman]

    The name ``treat-as-defined'' is not telling enough what is its
    effect.

It seems clear to me.

	     How about a more general facility, like this:

      (bytecomp-no-warn 'undefined-function FILE FUNC1 FUNC2 ...)

It is more cumbersome, and I don't see that we need any more
generality.

Re: byte compiler warnings when bootstrapping

[Stefan Monnier]

>       (require-lazily <feature>)

>     which would behave similarly to `require' except that the byte-compiler
>     would replace it with a bunch of autoloads.  so the package would only be
>     "required" when one of its functions was used.

> I have a bad feeling about that -- that it would produce scads of autoloads
> which are not needed.  Also, I don't think it is needed in any of the existing
> cases of warnings we would like to suppress.

The intention is for the byte-compiler to take the intersection of the
functions defined by <feature> and the functions used in the file being
byte-compiled.

So I expect that in general it will not lead to tons of autoloads that are
not needed.  Of course, some of the autoloads will not be needed because
some of those functions will only ever be called after calling another one
of those autoloaded functions (like the example of calling
info-read-node-name after calling `info').  And actually, the byte-compiler
could even optimize away some of those cases.


        Stefan

Re: byte compiler warnings when bootstrapping

[Stefan Monnier]

>     I think it would be good.  Although I think I'd rather allow the
>     external-function-delcaration to give an arglist so that the bytecompiler
>     can check you passed the right number of arguments as well.  So maybe
>     something like

>       (declare-function <foo> <fooarglist>)

> That seems like a good design, but I think it should also have
> the file name, so that something at build time can check all these
> against the files they refer to.

I don't think the <file> argument is important (the build-time check can
just as well read all the files to figure out whether the function is
defined in some file), but it wouldn't hurt, indeed.


        Stefan

Re: byte compiler warnings when bootstrapping

[Richard Stallman]

    The intention is for the byte-compiler to take the intersection of the
    functions defined by <feature> and the functions used in the file being
    byte-compiled.

This requires it to know which functions are defined by that feature.
In other words, compilation would have to USE the table produced
by the batch process.

I prefer the approach I designed, in which the batch process finds
erroneous declarations but its output is not needed to compile
anything.

Re: byte compiler warnings when bootstrapping

[Stefan Monnier]

>     The intention is for the byte-compiler to take the intersection of the
>     functions defined by <feature> and the functions used in the file being
>     byte-compiled.

> This requires it to know which functions are defined by that feature.
> In other words, compilation would have to USE the table produced
> by the batch process.

Well there would be no separate batch process for it: it's only a change in
the byte-compiler which will treate require-lazy as a normal require during
compilation (i.e. it will load the file during compilation) but will replace
the `require-lazy' by a bunch of autoloads in the .elc file.

It's a different feature than the one you imagine.  Both might be useful.


        Stefan

Re: byte compiler warnings when bootstrapping

[Richard Stallman]

    Well there would be no separate batch process for it: it's only a change in
    the byte-compiler which will treate require-lazy as a normal require during
    compilation (i.e. it will load the file during compilation) but will replace
    the `require-lazy' by a bunch of autoloads in the .elc file.

Thanks for clarifying, but I think it is a drawback that this needs to
load the other file during compilation.