"reference to free variable" only during initialization

"reference to free variable" only during initialization

[Yuri D'Elia]

I have an oddball warning that I have an hard time debugging. I upgraded
to emacs 25.1.1 on a debian desktop recently, and started to get:

Warning (bytecomp): reference to free variable ‘isearchp-reg-beg’

during startup. The warning is generated by requiring
modeline-posn.el[1], which is using ``isearchp-reg-beg`` (actually
defvar-ed early on) inside a defadvice form.

Is this expected?

Warning aside, the intriguing part is that the warning /only/ occurs
when (require 'modeline-posn) is evaluated in my ~/.emacs, but is not
emitted instead when doing the "allegedly" equivalent:

  emacs -q -l ~/.emacs

even when ~/.emacs itself only contains:

  (add-to-list 'load-path "path-to-modeline.el")
  (require 'modeline-posn)

or when simply doing an eval in a fresh emacs -q session

The remaining difference is that emacs -q is skipping debian's system
startup. But I cannot think of something that could influence the
generation of a bytecomp warning.

Any pointer before going further?

[1] https://www.emacswiki.org/emacs/modeline-posn.el

Re: "reference to free variable" only during initialization

[Noam Postavsky]

On Mon, Jan 16, 2017 at 6:00 PM, Yuri D'Elia <wavexx@thregr.org> wrote:
> I have an oddball warning that I have an hard time debugging. I upgraded
> to emacs 25.1.1 on a debian desktop recently, and started to get:
>
> Warning (bytecomp): reference to free variable ‘isearchp-reg-beg’
>
> during startup. The warning is generated by requiring
> modeline-posn.el[1], which is using ``isearchp-reg-beg`` (actually
> defvar-ed early on) inside a defadvice form.
>
> Is this expected?

There is some unexpected interaction of defadvice with delayed
warnings. I had noticed this a while back, but never got around to
fixing it. `ad-compile-function' binds `warning-suppress-types', but
if warnings are delayed, then the warning call happens after that
binding is finished.

>
> Warning aside, the intriguing part is that the warning /only/ occurs
> when (require 'modeline-posn) is evaluated in my ~/.emacs, but is not
> emitted instead when doing the "allegedly" equivalent:
>
>   emacs -q -l ~/.emacs

In this case, the warnings are not delayed. If you set after-init-time
to nil, then warnings will be delayed, so the same problem should
occur, I think:

    emacs -q --eval '(setq after-init-time nil)' -l ~/.emacs

I think Bug #24106 might be a similar case (and I'm pretty sure
there's another older one where I initially noticed this, but I can't
find it right now).

Re: "reference to free variable" only during initialization

[Yuri D'Elia]

On Mon, Jan 16 2017, Noam Postavsky wrote:
>> Warning (bytecomp): reference to free variable ‘isearchp-reg-beg’
>>
>> during startup. The warning is generated by requiring
>> modeline-posn.el[1], which is using ``isearchp-reg-beg`` (actually
>> defvar-ed early on) inside a defadvice form.
>>
>> Is this expected?
>
> There is some unexpected interaction of defadvice with delayed
> warnings. I had noticed this a while back, but never got around to
> fixing it. `ad-compile-function' binds `warning-suppress-types', but
> if warnings are delayed, then the warning call happens after that
> binding is finished.

This explains one difference, but it would mean that when the warning is
not delayed, it would be lost[!]. That would be more serious.

I assumed instead that defadvice was evaluating the body too early,
before defvar (and thus generating an /incorrect/ warning).

>>   emacs -q -l ~/.emacs
>
> In this case, the warnings are not delayed. If you set after-init-time
> to nil, then warnings will be delayed, so the same problem should
> occur, I think:
>
>     emacs -q --eval '(setq after-init-time nil)' -l ~/.emacs

Actually, it doesn't work. No warning is generated in this case.

But if I do the opposite: set after-init-time in the beginning of my
~/.emacs, then I can suppress the warning which indeed suggests some
problem with the delaying.

Re: "reference to free variable" only during initialization

[Noam Postavsky]

On Wed, Jan 25, 2017 at 12:15 PM, Yuri D'Elia <wavexx@thregr.org> wrote:
> On Mon, Jan 16 2017, Noam Postavsky wrote:
>>> Warning (bytecomp): reference to free variable ‘isearchp-reg-beg’
>>>
>>> during startup. The warning is generated by requiring
>>> modeline-posn.el[1], which is using ``isearchp-reg-beg`` (actually
>>> defvar-ed early on) inside a defadvice form.
>>>
>>> Is this expected?
>>
>> There is some unexpected interaction of defadvice with delayed
>> warnings. I had noticed this a while back, but never got around to
>> fixing it. `ad-compile-function' binds `warning-suppress-types', but
>> if warnings are delayed, then the warning call happens after that
>> binding is finished.
>
> This explains one difference, but it would mean that when the warning is
> not delayed, it would be lost[!]. That would be more serious.

It shouldn't be lost, `warning-suppress-types' just stops the
*Warnings* buffer from popping up. The warnings are still logged in
the background.

>> In this case, the warnings are not delayed. If you set after-init-time
>> to nil, then warnings will be delayed, so the same problem should
>> occur, I think:
>>
>>     emacs -q --eval '(setq after-init-time nil)' -l ~/.emacs
>
> Actually, it doesn't work. No warning is generated in this case.

Hmm, I can't reproduce your original case in 25.1 or 24.5, even after commenting

(defvar isearchp-reg-beg)                 ; In `isearch+.el'
(defvar isearchp-reg-end)                 ; In `isearch+.el'
(defvar isearchp-restrict-to-region-flag) ; In `isearch+.el'

at the top of modeline-posn.el. I have init.el with contents

Re: "reference to free variable" only during initialization

[Noam Postavsky]

Hit send too early.

On Wed, Jan 25, 2017 at 2:50 PM, Noam Postavsky
<npostavs@users.sourceforge.net> wrote:
> at the top of modeline-posn.el. I have init.el with contents

(add-to-list 'load-path "~/.emacs.d/lisp")
(require 'modeline-posn)

when I start emacs normally I see no warnings.

Re: "reference to free variable" only during initialization

[Yuri D'Elia]

On Wed, Jan 25 2017, Noam Postavsky wrote:
> Hit send too early.
>
> On Wed, Jan 25, 2017 at 2:50 PM, Noam Postavsky
> <npostavs@users.sourceforge.net> wrote:
>> at the top of modeline-posn.el. I have init.el with contents
>
> (add-to-list 'load-path "~/.emacs.d/lisp")
> (require 'modeline-posn)
>
> when I start emacs normally I see no warnings.

Sorry for the late reply, but I could find some time just now. I tried
to reproduce this on a fresh emacs (from the current master - 26+).

This is the resulting minimal ~/.emacs.d/init.el:

  (package-initialize)
  (add-to-list 'load-path "~/.emacs.d/lisp")
  (require 'modeline-posn)

ran with:

  emacs --no-site-file --no-site-lisp --no-splash --no-x-resources

with ~/.emacs.d/lisp containing modeline-posn.el from melpa. There's no
other file. I indeed had to add (package-initialize) to reproduce the
issue, but I'm not sure why it has some influence. Maybe just some
delay.

Right off the bat, could you reproduce it with this?

Evaling after-init-time makes no difference, and I also couldn't
reproduce it with my own directions I gave beforehand.

Re: "reference to free variable" only during initialization

[Noam Postavsky]

On Sat, Feb 18, 2017 at 6:01 PM, Yuri D'Elia <wavexx@thregr.org> wrote:
> with ~/.emacs.d/lisp containing modeline-posn.el from melpa. There's no
> other file. I indeed had to add (package-initialize) to reproduce the
> issue, but I'm not sure why it has some influence. Maybe just some
> delay.
>
> Right off the bat, could you reproduce it with this?

Yes, the reason package-initialize matters is just that it loads
bytecomp and advice.el only decides to compile if that has been
loaded:

    (defun ad-should-compile (function compile)
      (cond
       ...
       ;; everything else means `maybe':
       (t (featurep 'byte-compile))))

(bytecomp.el provides both `bytecomp' and `byte-compile').

>
> Evaling after-init-time makes no difference,

--eval '(setq after-init-time t)' has no effect, but doing

    (let ((after-init-time t))
      (require 'modeline-posn))

in the init.el file does prevent the warning from popping up. It also
improves the error message: the prefix becomes
".emacs.d/lisp/modeline-posn.el:Warning:" instead of "Warning
(bytecomp):". This is because byte compilation is let-binding
warning-prefix-function which is not saved for delayed warnings.

Re: "reference to free variable" only during initialization

[Yuri D'Elia]

On Sat, Feb 18 2017, Noam Postavsky wrote:
> --eval '(setq after-init-time t)' has no effect, but doing
>
>     (let ((after-init-time t))
>       (require 'modeline-posn))
>
> in the init.el file does prevent the warning from popping up. It also
> improves the error message: the prefix becomes
> ".emacs.d/lisp/modeline-posn.el:Warning:" instead of "Warning
> (bytecomp):". This is because byte compilation is let-binding
> warning-prefix-function which is not saved for delayed warnings.

Understood. But then, back to the original issue: can you explain the
warning? modeline-posn doesn't seem to be doing strange things here.

Re: "reference to free variable" only during initialization

[Noam Postavsky]

On Sun, Feb 19, 2017 at 7:32 AM, Yuri D'Elia <wavexx@thregr.org> wrote:
>
> Understood. But then, back to the original issue: can you explain the
> warning? modeline-posn doesn't seem to be doing strange things here.
>

At the end of the (defadvice isearch-query-replace-regexp ...) there is

    `(lambda () (setq modelinepos-region-acting-on  ',isearchp-reg-beg))

This uses isearch-reg-beg without checking if it's boundp (the other
occurences do have a boundp check first).

Re: "reference to free variable" only during initialization

[Yuri D'Elia]

On Sun, Feb 19 2017, Noam Postavsky wrote:
> At the end of the (defadvice isearch-query-replace-regexp ...) there is
>
>     `(lambda () (setq modelinepos-region-acting-on  ',isearchp-reg-beg))
>
> This uses isearch-reg-beg without checking if it's boundp (the other
> occurences do have a boundp check first).

Ah! Well, this is interesting, I completely missed this usage. And after
understanding it, I'm also somehow surprised that this is not showing up
more often.

RE: "reference to free variable" only during initialization

[Drew Adams]

> > `(lambda () (setq modelinepos-region-acting-on  ',isearchp-reg-beg))
> > This uses isearch-reg-beg without checking if it's boundp (the other
> > occurences do have a boundp check first).
> 
> Ah! Well, this is interesting, I completely missed this usage. And after
> understanding it, I'm also somehow surprised that this is not showing up
> more often.

Thanks to Yuri for reporting the original problem and to Noam for
finding that missing boundp occurrence.  (Should be OK now.)