bug#55137: Different result when interpreted and when evaluating byte-compiled code

[Paul Pogonyshev <pogonyshev@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 2269 bytes --]

I'm not absolutely sure if it is a bug or a "feature", but it is
extremely confusing.

To reproduce: store this as file `test.el':

    (defvar special-variable nil)

    (defmacro is-special-as-macro ()
      (special-variable-p 'special-variable))

    (defun is-special-as-function ()
      (is-special-as-macro))

    (print (is-special-as-function))
    (print (eval '(is-special-as-macro)))

Now, from command line:

    $ rm -f test.elc; emacs --batch -l test.el

This loads the file as interpreted Elisp and prints `t' two times,
i.e. always recognizes the variable as special.

Next, byte-compile the file before loading:

    $ rm -f test.elc; emacs --batch --eval "(byte-compile-file
\"test.el\")"; emacs --batch -l test.elc

This prints `nil' and `t', i.e. variable is now considered non-special
by `is-special-as-function'.  From investigating `.elc' file, it is
apparent that this is encoded into it as a macroexpanded constant.
Note that `is-special-as-macro' still works fine, the problem appears
only when it gets macroexpanded during byte-compilation.

So, apparently `defvar' form is sort of "skipped without paying
attention" during byte-compilation.  If I put it into an
`eval-and-compile' form, then macroexpansion does produce expected
result.

I noticed this with code using `iter2' library.  When `iter2-defun'
generator functions get macroexpanded during compilation, built-in
`special-variable-p' is called, because for generator functions this
is important to distinguish between local and dynamic variables.  The
example above is derived from debugging real failure.

Standard `generator' package is also affected.  Here is example code:

    ;;; -*- lexical-binding: t -*-

    (require 'generator)

    (defvar special-variable nil)

    (defun get-special-variable ()
      special-variable)

    (iter-defun buggy ()
      (let ((special-variable t))
        (iter-yield (get-special-variable))))

    (print (iter-next (buggy)))

As before, save as `test.el' and execute the same two commands.
Produced output is different depending on whether the file has been
byte-compiled or not.

Is `eval-and-compile' the proper workaround?  Can things be made less
confusing by noticing declared special variables during
byte-compilation?

Paul

[-- Attachment #2: Type: text/html, Size: 2677 bytes --]