Expansion of #$ in byte-compiled files

Expansion of #$ in byte-compiled files

[Michael Mauger]

Please correct me if I am not right here, but it appears that in byte-compiled files (don't always?) expand #$ to the current elisp file name.

I wrote a module containing the following code:

  (message "x: %S" #$)

When I load this module using load-file of the .el file adds a line to the *Message* buffer

  Loading /user/michael/x.el (source)...

  x: "/user/michael/x.el"
  Loading /user/michael/x.el (source)...done


I then byte-compile the module, use load-file on the .elc file, and now I get 


  Loading /user/michael/x.elc...
  x: nil
  Loading /user/michael/x.elc...done


This causes all loading ELPA/MELPA packages to generate the following error during initialization:

  (wrong-type-argument stringp nil)

because of the following generated line in the package autoloads script:

  (add-to-list 'load-path (or (file-name-directory #$) (car load-path))) 

and then the autoloads file gets byte-compile'd when installed.

This is with the latest Cygwin version with the W32 GUI:

  GNU Emacs 24.5.1 (x86_64-unknown-cygwin)
   of 2015-06-23 on desktop-new

Are others seeing the issue? Is this cygwin specific?  I will try this out on a Linux latest dev version as soon as I can.

Re: Expansion of #$ in byte-compiled files

[Ken Brown]

On 9/3/2015 3:53 PM, Michael Mauger wrote:
> Please correct me if I am not right here, but it appears that in byte-compiled files (don't always?) expand #$ to the current elisp file name.
>
> I wrote a module containing the following code:
>
>    (message "x: %S" #$)
>
> When I load this module using load-file of the .el file adds a line to the *Message* buffer
>
>    Loading /user/michael/x.el (source)...
>
>    x: "/user/michael/x.el"
>    Loading /user/michael/x.el (source)...done
>
>
> I then byte-compile the module, use load-file on the .elc file, and now I get
>
>
>    Loading /user/michael/x.elc...
>    x: nil
>    Loading /user/michael/x.elc...done
>
>
> This causes all loading ELPA/MELPA packages to generate the following error during initialization:
>
>    (wrong-type-argument stringp nil)
>
> because of the following generated line in the package autoloads script:
>
>    (add-to-list 'load-path (or (file-name-directory #$) (car load-path)))
>
> and then the autoloads file gets byte-compile'd when installed.

Why does the autoloads file get byte-compiled?  All the package 
autoloads files on my system have the following in the "Local Variables" 
section:

    ;; no-byte-compile: t

Ken

Re: Expansion of #$ in byte-compiled files

[Kalle Olavi Niemitalo]

Michael Mauger <michael@mauger.com> writes:

> because of the following generated line in the package autoloads script:
>
>   (add-to-list 'load-path (or (file-name-directory #$) (car load-path))) 

I expect that (file-name-directory load-file-name) would avoid
the problem.

Re: Expansion of #$ in byte-compiled files

[Michael Mauger]

> On Thursday, September 3, 2015 5:51 PM, Kalle Olavi Niemitalo <kon@iki.fi> wrote:

> > Michael Mauger <michael@mauger.com> writes:
> 
> 
>>  because of the following generated line in the package autoloads script:
>> 
>>    (add-to-list 'load-path (or (file-name-directory #$) (car 
> load-path))) 
> 
> I expect that (file-name-directory load-file-name) would avoid
> the problem.> 

Yes, using`load-file-name' might avoid the problem but the package installer generates the #$ syntax so I have no direct control of it.

Ken Brown was correct that the autoloads.el files have a `no-byte-compile: t' entry in their Local Variables section, but the package installer is still byte compiling the files.  I think the placement of some ^L are interfering with the proper parsing of that section.  The good news is that it is not happening in the master repository, only on my 25.4 Cygwin instance.  I have added a snippet of code to delete any *-autoloads.elc files in the package tree before I initialize the package system and that avoids the problem.

I checked my latest GIT version on Linux and confirmed it has no compiled autoloads and installing new packages was not creating them.  I then double checked the update of existing packages and installation of new packages did compile the autoloads on my W7/Cygwin install.


My problem is fixed and hopefully future users of 25.4 will understand the problem if they encounter it.  Feel free to ask me anything that might help further debug the issue if you'd like; but, again, it does appear to be fixed.

Re: Expansion of #$ in byte-compiled files

[Ken Brown]

On 9/3/2015 9:46 PM, Michael Mauger wrote:
>> On Thursday, September 3, 2015 5:51 PM, Kalle Olavi Niemitalo <kon@iki.fi> wrote:
>
>>> Michael Mauger <michael@mauger.com> writes:
>>
>>
>>>   because of the following generated line in the package autoloads script:
>>>
>>>     (add-to-list 'load-path (or (file-name-directory #$) (car
>> load-path)))
>>
>> I expect that (file-name-directory load-file-name) would avoid
>> the problem.>
>
> Yes, using`load-file-name' might avoid the problem but the package installer generates the #$ syntax so I have no direct control of it.
>
> Ken Brown was correct that the autoloads.el files have a `no-byte-compile: t' entry in their Local Variables section, but the package installer is still byte compiling the files.  I think the placement of some ^L are interfering with the proper parsing of that section.  The good news is that it is not happening in the master repository, only on my 25.4 Cygwin instance.  I have added a snippet of code to delete any *-autoloads.elc files in the package tree before I initialize the package system and that avoids the problem.
>
> I checked my latest GIT version on Linux and confirmed it has no compiled autoloads and installing new packages was not creating them.  I then double checked the update of existing packages and installation of new packages did compile the autoloads on my W7/Cygwin install.

I just tested this, using the same emacs version as you (the current 
release of emacs-w32 on 64-bit Cygwin), and the autoloads files didn't 
get byte-compiled.  I tested by installing form-feed and lua-mode from 
MELPA.  I don't know what could be different on your system to cause 
this problem.  Can you reproduce it from 'emacs -Q'?

Ken

Re: Expansion of #$ in byte-compiled files

[Stefan Monnier]

> Please correct me if I am not right here, but it appears that in
> byte-compiled files (don't always?) expand #$ to the current elisp
> file name.  I wrote a module containing the following code:

>   (message "x: %S" #$)

The issue here is that #$ is a "reader macro", so it is replaced by the
content of load-file-name when this expression is *read*.  When the .el
file is loaded, this expression is read when load-file-name has the
intended value, but when it is compiled, this expression is read during
byte-compilation (i.e. not during any kind of "load"ing) when
load-file-name is nil.  So #$ will be read as if nil appeared there in
the file.  Hence the byte-compiler will generate code which behaves just like

   (message "x: %S" nil)


-- Stefan