Emacs 25.0.94: Is require failing to define macros and functions at compile time?

Emacs 25.0.94: Is require failing to define macros and functions at compile time?

[Robert Weiner]

I have been seeing a number of problems with byte-compiling the
Hyperbole package due to require statements (probably within
byte-compiled files) not defining certain macros and functions needed
during the compilation of the file with the require in it.  If the
required file is named F, then (require 'F) is insufficient but
(eval-when-compile (load "F.el")) works  (a load of "F" does not
always work), so I am forced to add these eval-when-compile statements
to the code.

Has anyone seen anything similar?  Most of the time the problem is
with a macro definition not be defined and therefore a macro expansion
fails, but sometimes I get undefined functions when I can see the
function is defined within the required file and I know that the path
of the file is in load-path at compile time.  Sometimes the problem is
with recursive requires.

Has anyone seen anything similar?

Isn't require supposed to be sufficient to define both macros and
functions at compile time for a client library?

Here is a recent example from Hyperbole:

When compiling hact.el, I get:
In end of data:
hact.el:322:1:Warning: the following functions are not known to be defined:
    hhist:add

At the top of hact.el is:

(mapc 'require '(hhist set))

In hhist.el is:

(defun hhist:add (elt) ...)

Something is not happening at byte-compile time but why and what is the problem?

Bob

Re: Emacs 25.0.94: Is require failing to define macros and functions at compile time?

[Clément Pit--Claudel]

[-- Attachment #1.1: Type: text/plain, Size: 304 bytes --]

On 2016-06-29 10:55, Robert Weiner wrote:
> At the top of hact.el is:
> 
> (mapc 'require '(hhist set))

Try 

  (eval-and-compile (mapc 'require '(hhist set)))

Or simply

  (require 'hhist)
  (require 'set)

My guess would be that the byte-compiler doesn't run your mapc.

Clément.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

RE: Emacs 25.0.94: Is require failing to define macros and functions at compile time?

[Drew Adams]

> At the top of hact.el is:
> (mapc 'require '(hhist set))

I think that's your problem.  The byte-compiler is probably
looking for a literal (require 'hhist) and (require 'set).

The Elisp manual, node `Named Features', says this:

   When ‘require’ is used at top level in a file, it takes effect when
 you byte-compile that file (*note Byte Compilation::) as well as when
 you load it.  This is in case the required package contains macros that
 the byte compiler must know about.  It also avoids byte compiler
 warnings for functions and variables defined in the file loaded with
 ‘require’.

You might be able to get away with wrapping that `mapc' in an
`eval-when-compile'.  But why bother with the `mapc' at all?

Just a guess - someone else will correct me if wrong.

Re: Emacs 25.0.94: Is require failing to define macros and functions at compile time?

[Robert Weiner]

On Wed, Jun 29, 2016 at 11:14 AM, Clément Pit--Claudel
<clement.pit@gmail.com> wrote:
> My guess would be that the byte-compiler doesn't run your mapc.

That sounds like it.  Thanks, I will fix the usage.

So maybe the byte-compiler could be made smarter to handle a list of
requires.  If a library has 10 requires, why should we have to write
out require 10 times or add the eval-and-compile call?  Maybe there is
a need for a require-list primitive for such circumstances so that it
is always available at compilation time.

Bob

Re: Emacs 25.0.94: Is require failing to define macros and functions at compile time?

[Davis Herring]

> So maybe the byte-compiler could be made smarter to handle a list of
> requires.  If a library has 10 requires, why should we have to write
> out require 10 times or add the eval-and-compile call?  Maybe there is
> a need for a require-list primitive for such circumstances so that it
> is always available at compilation time.

Any library complicated enough to have N `require's will by necessity 
have so many more lines of code than N that the longhand list of 
(require 'foo) will be negligible.  Doing it the simple(minded), obvious 
way has the advantage that it makes trivial things like grep easier.

Obviously it is a complexity that the meaning of `require' depends on 
whether you're compiling or not; the above is simply a statement about 
practicalities.

Davis

-- 
This product is sold by volume, not by mass.  If it appears too dense or 
too sparse, it is because mass-energy conversion has occurred during 
shipping.

RE: Emacs 25.0.94: Is require failing to define macros and functions at compile time?

[Drew Adams]

> Any library complicated enough to have N `require's will by necessity
> have so many more lines of code than N that the longhand list of
> (require 'foo) will be negligible.  Doing it the simple(minded), obvious
> way has the advantage that it makes trivial things like grep easier.

+1

(Also, it's more readable by humans, IMO.)

Re: Emacs 25.0.94: Is require failing to define macros and functions at compile time?

[Robert Weiner]

On Wed, Jun 29, 2016 at 3:59 PM, Drew Adams <drew.adams@oracle.com> wrote:
>> Any library complicated enough to have N `require's will by necessity
>> have so many more lines of code than N that the longhand list of
>> (require 'foo) will be negligible.

What happened to less code is easier to maintain?  Or don't repeat yourself?

> Doing it the simple(minded), obvious
> way has the advantage that it makes trivial things like grep easier.

I do:

grep "require.*<my-library>" *.el

all the time without a problem on libraries that use the (mapc
#'require ...) call, so it is not a problem.

Bob

RE: Emacs 25.0.94: Is require failing to define macros and functions at compile time?

[Drew Adams]

> On Wed, Jun 29, 2016 at 3:59 PM, Drew Adams <drew.adams@oracle.com> wrote:
>
> >> Any library complicated enough to have N `require's will by necessity
> >> have so many more lines of code than N that the longhand list of
> >> (require 'foo) will be negligible.

No, I didn't write that.  (But I did second it.)

> What happened to less code is easier to maintain?  Or don't repeat yourself?

Platitudinous abstraction.  Devil is in details.

Re: Emacs 25.0.94: Is require failing to define macros and functions at compile time?

[John Mastro]

Robert Weiner <rsw@gnu.org> wrote:
> What happened to less code is easier to maintain?  Or don't repeat yourself?

The benefit of less code and "DRY" is that they make systems easier to
comprehend, maintain, and extend. Those are very important properties
but arguably (and IMO) don't apply to this case (or, IMO, to the earlier
discussion regarding `called-interactively-p').

Obviously, this is subjective and arguably bikeshedding. I like less
code too - I just don't think it will always and in every case lead us
to the best overall result.

        John

Re: Emacs 25.0.94: Is require failing to define macros and functions at compile time?

[Michael Heerdegen]

Drew Adams <drew.adams@oracle.com> writes:

> You might be able to get away with wrapping that `mapc' in an
> `eval-when-compile'.  But why bother with the `mapc' at all?

Presumably `eval-and-compile', I think, since he also wants to call the
`require' statements when loading the file.

Michael.

Re: Emacs 25.0.94: Is require failing to define macros and functions at compile time?

[Michael Heerdegen]

Robert Weiner <rsw@gnu.org> writes:

> Most of the time the problem is with a macro definition not be defined
> and therefore a macro expansion fails, but sometimes I get undefined
> functions when I can see the function is defined within the required
> file and I know that the path of the file is in load-path at compile
> time.

If the thing is a macro, loading the defining file after byte compiling
is too late, the interpreter then only looks for function definitions,
so you'll get the error of an undefined function as you saw it.


Michael.

Re: Emacs 25.0.94: Is require failing to define macros and functions at compile time?

[Robert Weiner]

On Wed, Jun 29, 2016 at 6:03 PM, Michael Heerdegen
<michael_heerdegen@web.de> wrote:
> Presumably `eval-and-compile', I think, since he also wants to call the
> `require' statements when loading the file.

Yes, that solved things nicely.

On a related note, I like to define variables private to a library at
the end of the library after all functions.  But then the
byte-compiler gives warnings that I am referencing free variables
since it doesn't yet know about the definition.  Is there a way to
have it load the .el version of the file first and then byte-compile
it if that is what I want?

Thanks,

Bob

Re: Emacs 25.0.94: Is require failing to define macros and functions at compile time?

[Michael Heerdegen]

Robert Weiner <rsw@gnu.org> writes:

> Is there a way to have it load the .el version of the file first and
> then byte-compile it if that is what I want?

Sure (there is always a way in Emacs...), but I don't think it's a good
idea to use such a hack just for a more cosmetic purpose.

IIRC the currently compiled file can be referred to with the variable
`byte-compile-current-file', even from the file itself.  I don't recall
the downsides of loading the file while compiling (apart from making
compiling slow and loading the package unnecessarily when compiling).


Michael.

RE: Emacs 25.0.94: Is require failing to define macros and functions at compile time?

[Drew Adams]

> > Is there a way to have it load the .el version of the file first and
> > then byte-compile it if that is what I want?
> 
> Sure (there is always a way in Emacs...), but I don't think it's a good
> idea to use such a hack just for a more cosmetic purpose.
> 
> IIRC the currently compiled file can be referred to with the variable
> `byte-compile-current-file', even from the file itself.  I don't recall
> the downsides of loading the file while compiling (apart from making
> compiling slow and loading the package unnecessarily when compiling).

From (elisp) `Named Features':

   When 'require' is used at top level in a file, it takes effect when
 you byte-compile that file (*note Byte Compilation::) as well as when
 you load it.  This is in case the required package contains macros that
 the byte compiler must know about.  It also avoids byte compiler
 warnings for functions and variables defined in the file loaded with
 'require'.

   Although top-level calls to 'require' are evaluated during byte
 compilation, 'provide' calls are not.  Therefore, you can ensure that a
 file of definitions is loaded before it is byte-compiled by including a
 'provide' followed by a 'require' for the same feature, as in the
 following example.

     (provide 'my-feature)  ; Ignored by byte compiler,
                            ;   evaluated by 'load'.
     (require 'my-feature)  ; Evaluated by byte compiler.

 The compiler ignores the 'provide', then processes the 'require' by
 loading the file in question.  Loading the file does execute the
 'provide' call, so the subsequent 'require' call does nothing when the
 file is loaded.

Re: Emacs 25.0.94: Is require failing to define macros and functions at compile time?

[Robert Weiner]

On Wed, Jun 29, 2016 at 6:42 PM, Drew Adams <drew.adams@oracle.com> wrote:
>      (provide 'my-feature)  ; Ignored by byte compiler,
>                             ;   evaluated by 'load'.
>      (require 'my-feature)  ; Evaluated by byte compiler.
>
>  The compiler ignores the 'provide', then processes the 'require' by
>  loading the file in question.  Loading the file does execute the
>  'provide' call, so the subsequent 'require' call does nothing when the
>  file is loaded.

Thank you for the pointer.

I would, however, rather have a byte-compiler option that can be set
once per package that produces this behavior so I don't have to do
this on a per-file basis or even look at any code for it each time I
read the source, just like the byte-compile-warnings variable lets me
do away with warnings I don't need in one central location.

Bob

RE: Emacs 25.0.94: Is require failing to define macros and functions at compile time?

[Herring, Davis]

> What happened to less code is easier to maintain?  Or don't repeat yourself?

Asymptotically it is just a constant factor fewer characters to use mapc here.  And (require 'foo) forms are so trivial that the maintenance burden is questionable; some would prefer the diff- (and blame-) friendly character of having each on its own line.  As for DRY, the various libraries are typically not strongly related (or else they would be combined), so the need for each is defensibly separate.

> I do:
> 
> grep "require.*<my-library>" *.el
> 
> all the time without a problem on libraries that use the (mapc
> #'require ...) call, so it is not a problem.

If all the libraries fit on one line, yes.  If there are so few, why bother with mapc anyway?

I'm not making any claim that these are "crushing rhetorical blows"; merely pointing out that it's (probably) not worth making the world more complicated for the sake of curing so mild a problem as this.

Davis

Re: Emacs 25.0.94: Is require failing to define macros and functions at compile time?

[Robert Weiner]

On Wed, Jun 29, 2016 at 8:09 PM, Herring, Davis <herring@lanl.gov> wrote:
> If all the libraries fit on one line, yes.  If there are so few, why bother with mapc anyway?

It reads better to me, just as (+ 2 3 4 5 6) reads better to me than
(2 + 3 + 4 + 5 + 6).  My brain sees and processes one operator and
then handles the operands.  I guess I just see things a bit
differently than many others (I find CamelCase distressingly hard to
read yet people write it everyday; I think user interfaces are fun to
program and should be both highly usable and powerful).

> I'm not making any claim that these are "crushing rhetorical blows"; merely pointing out that it's (probably) not worth making the world
> more complicated for the sake of curing so mild a problem as this.

I agree.  Now that I know the significance of top-level placement for
require and load during compilation, I can work with it and do
whatever needs to be done.

Bob