bug#49749: 26.3; 26.3 & 27.2: invalid byte compiler warning in short-circuited or form

bug#49749: 26.3; 26.3 & 27.2: invalid byte compiler warning in short-circuited or form

[Pierre Rouleau]

Paste the following code inside the scratch buffer (nothing else):

(defun f-and ()
  "Use and."
  (when (null (and (boundp 'foo)
                  foo))  ; no warning here, as expected.                                            
    (message "foo is not set")))

(defun f-or ()
  "Use or."
  (when (or (null (boundp 'foo))
            (null foo))  ;=> ``Warning: reference to free variable ‘foo’``                          
    (message "foo is not set")))

Then do:

- ``M-x compile-defun`` with point over f-and.
  No warning is reported as expected.

- ``M-x compile-defun`` with point over f-or.
  This time the byte compiler (of both Emacs 26.3 and 27.2) reports a
  "Warning: reference to free variable ‘foo’" on the line identified in
  the code.

I believe the byte compiler warning to be invalid.
The symbol `foo' is not bound but the code checks if it is
bound first with the line just above.
The ``or`` form is short-circuiting and will not eval
the (null foo) at that point.

Furthermore the code in `f-or' is a De Morgan equivalent boolean
expression of the one inside the `f-and' function.

Both functions `f-and' and `f-or' will run to completion when called and
both issue the "foo is not set" message.



In GNU Emacs 26.3 (build 1, x86_64-apple-darwin18.6.0)
of 2019-08-30 built on Mojave.local
Recent messages:
Contacting host: debbugs.gnu.org:443
Opening nndoc server on /var/folders/xb/xffhmkvs3ssfdqd4xk7lftsr0000gn/T/gnus-temp-group-6iUAmw-ephemeral...done
Contacting host: debbugs.gnu.org:443
Opening nndoc server on /var/folders/xb/xffhmkvs3ssfdqd4xk7lftsr0000gn/T/gnus-temp-group-Y4gIQl-ephemeral...done
Contacting host: debbugs.gnu.org:443
Opening nndoc server on /var/folders/xb/xffhmkvs3ssfdqd4xk7lftsr0000gn/T/gnus-temp-group-87ADIS-ephemeral...done
Contacting host: debbugs.gnu.org:443
Opening nndoc server on /var/folders/xb/xffhmkvs3ssfdqd4xk7lftsr0000gn/T/gnus-temp-group-MiDCIW-ephemeral...done
Contacting host: debbugs.gnu.org:443
Opening nndoc server on /var/folders/xb/xffhmkvs3ssfdqd4xk7lftsr0000gn/T/gnus-temp-group-puQ3A6-ephemeral...done

Configured using:
'configure --disable-dependency-tracking --disable-silent-rules
--enable-locallisppath=/usr/local/share/emacs/site-lisp
--infodir=/usr/local/Cellar/emacs/26.3/share/info/emacs
--prefix=/usr/local/Cellar/emacs/26.3 --with-gnutls --without-x
--with-xml2 --without-dbus --with-modules --without-ns
--without-imagemagick'

Configured features:
NOTIFY ACL GNUTLS LIBXML2 ZLIB MODULES THREADS

Important settings:
  value of $LANG: en_CA.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Debbugs

Minor modes in effect:
  eros-mode: t
  global-undo-tree-mode: t
  undo-tree-mode: t
  ido-everywhere: t
  which-key-mode: t
  winner-mode: t
  key-chord-mode: t
  global-anzu-mode: t
  anzu-mode: t
  show-paren-mode: t
  recentf-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
/Users/roup/.emacs.d/utils/benchmark-init hides /Users/roup/.emacs.d/elpa/benchmark-init-20150905.938/benchmark-init
/Users/roup/.emacs.d/utils/benchmark-init-modes hides /Users/roup/.emacs.d/elpa/benchmark-init-20150905.938/benchmark-init-modes
/Users/roup/.emacs.d/utils/sr-speedbar hides /Users/roup/.emacs.d/elpa/sr-speedbar-20161025.831/sr-speedbar
/Users/roup/.emacs.d/elpa/lispy-20210121.926/elpa hides /Users/roup/.emacs.d/elpa/ivy-20210311.1638/elpa
/Users/roup/.emacs.d/elpa/lfe-mode-20201007.2214/lfe-indent hides /usr/local/share/emacs/site-lisp/lfe/lfe-indent
/Users/roup/.emacs.d/elpa/lfe-mode-20201007.2214/lfe-mode hides /usr/local/share/emacs/site-lisp/lfe/lfe-mode
/Users/roup/.emacs.d/elpa/lfe-mode-20201007.2214/inferior-lfe hides /usr/local/share/emacs/site-lisp/lfe/inferior-lfe
/Users/roup/.emacs.d/elpa/soap-client-3.2.0/soap-client hides /usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/net/soap-client
/Users/roup/.emacs.d/elpa/soap-client-3.2.0/soap-inspect hides /usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/net/soap-inspect
/Users/roup/.emacs.d/elpa/let-alist-1.0.6/let-alist hides /usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/emacs-lisp/let-alist
/Users/roup/.emacs.d/elpa/seq-2.22/seq hides /usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/emacs-lisp/seq
/Users/roup/.emacs.d/elpa/nadvice-0.3/nadvice hides /usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/emacs-lisp/nadvice

Features:
(shadow emacsbug sendmail eieio-opt eros smex pel-numkpad pel-ccp
ibuf-ext ibuffer ibuffer-loaddefs windmove shr-color shr svg dom
browse-url flow-fill sort gnus-cite mail-extr gnus-async gnus-bcklg qp
gnus-agent gnus-srvr gnus-score score-mode nnvirtual nntp gnus-ml
gnus-msg disp-table nndoc gnus-cache gnus-dup gnus-art mm-uu mml2015
mm-view mml-smime smime dig pel-undo undo-tree diff misearch
multi-isearch mm-archive network-stream starttls url-cache org-rmail
org-mhe org-irc org-info org-gnus nnir gnus-sum gnus-group gnus-undo
gnus-start gnus-cloud nnimap nnmail mail-source utf7 netrc nnoo
parse-time gnus-spec gnus-int gnus-range message rfc822 mml mml-sec epa
derived epg mailabbrev gmm-utils mailheader gnus-win gnus nnheader
gnus-util rmail rmail-loaddefs mail-utils org-docview doc-view jka-compr
image-mode org-bibtex bibtex org-bbdb org-w3m crm ido-completing-read+
memoize cus-edit cus-start cus-load minibuf-eldef debbugs-org
org-element avl-tree generator org org-macro org-footnote org-pcomplete
pcomplete org-list org-faces org-entities org-version ob-emacs-lisp ob
ob-tangle org-src ob-ref ob-lob ob-table ob-keys ob-exp ob-comint
ob-core ob-eval org-compat org-macs org-loaddefs format-spec cal-menu
calendar cal-loaddefs debbugs-gnu add-log debbugs soap-client mm-decode
mm-bodies mm-encode url-http tls gnutls url-auth mail-parse rfc2231
rfc2047 rfc2045 mm-util ietf-drums mail-prsvr url-gw nsm rmc puny url
url-proxy url-privacy url-expand url-methods url-history url-cookie
url-domsuf url-util mailcap warnings rng-xsd rng-dt rng-util xsd-regexp
xml ace-link pel-skels-generic counsel xdg dired-x dired dired-loaddefs
compile comint ansi-color ido-grid ido pel-completion pel-seq which-key
indent-tools yafolding indent-tools-indentation-of winner pel-xref
pel-text-transform pel-read pel-navigate pel-scroll key-seq
pel-key-chord key-chord anzu term/xterm xterm paren tempo
pel-skels-elisp pel-text-insert pel-window pel-tempo pel-skels
pel-prompt lispy pcase swiper cl-extra ivy flx ivy-faces ivy-overlay
colir color delsel lispy-inline subr-x thingatpt avy noutline outline
easy-mmode etags xref project edebug lispy-tags mode-local find-func
pel__hydra hydra lv pel-lispy regexp-opt flyspell pel-spell ispell
cap-words superword subword imenu+ pel-imenu imenu elec-pair pel_keys
recentf tree-widget wid-edit speedbar sb-image ezimage image dframe
ls-lisp time-date delight pel-autoload pel--keys-macros pel--options
pel--macros pel--base pel finder-inf geiser-impl help-fns radix-tree
help-mode geiser-custom geiser-base ring info tool-bar edmacro kmacro
package easymenu epg-config url-handlers url-parse auth-source cl-seq
eieio eieio-core cl-macs eieio-loaddefs password-cache url-vars seq
byte-opt gv bytecomp byte-compile cconv benchmark-init advice
cl-loaddefs cl-lib mule-util tooltip eldoc electric uniquify ediff-hook
vc-hooks lisp-float-type tabulated-list replace newcomment text-mode
elisp-mode lisp-mode prog-mode register page menu-bar rfn-eshadow
isearch timer select mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite charscript charprop case-table epa-hook jka-cmpr-hook
help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote threads kqueue multi-tty make-network-process emacs)

Memory information:
((conses 16 840205 372854)
(symbols 48 54699 97)
(miscs 40 697 2352)
(strings 32 229530 157598)
(string-bytes 1 6394735)
(vectors 16 59391)
(vector-slots 8 1074310 78064)
(floats 8 3460 2680)
(intervals 56 15506 2187)
(buffers 992 50))

bug#49749: 26.3; 26.3 & 27.2: invalid byte compiler warning in short-circuited or form

[Lars Ingebrigtsen]

Pierre Rouleau <prouleau001@gmail.com> writes:

> (defun f-or ()
>   "Use or."
>   (when (or (null (boundp 'foo))
>             (null foo))  ;=> ``Warning: reference to free variable ‘foo’``                          
>     (message "foo is not set")))

The message about invalid stuff is only discarded if Emacs is trivially
able to deduce that it'll never be evaluated -- and as you've found out,
it's easy to make that heuristic not be heeded (see
`byte-compile-maybe-guarded' for details).

So I'm not sure this is a bug -- Emacs can't determine all cases where
we won't be executing the code in question at compile time.

Anybody else think that this is something that should be improved?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#49749: 26.3; 26.3 & 27.2: invalid byte compiler warning in short-circuited or form

[Pierre Rouleau]

[-- Attachment #1: Type: text/plain, Size: 1510 bytes --]

On Wed, Jul 28, 2021 at 11:26 AM Lars Ingebrigtsen <larsi@gnus.org> wrote:

> Pierre Rouleau <prouleau001@gmail.com> writes:
>
> > (defun f-or ()
> >   "Use or."
> >   (when (or (null (boundp 'foo))
> >             (null foo))  ;=> ``Warning: reference to free variable
> ‘foo’``
> >     (message "foo is not set")))
>
> The message about invalid stuff is only discarded if Emacs is trivially
> able to deduce that it'll never be evaluated -- and as you've found out,
> it's easy to make that heuristic not be heeded (see
> `byte-compile-maybe-guarded' for details).
>

Thanks.  I must admit I do not know the byte compiler code much at this
point.


> So I'm not sure this is a bug -- Emacs can't determine all cases where
> we won't be executing the code in question at compile time.
>

Well, from the perspective of a user, that would look at the
very least as a technical limitation.

The byte compiler is able to report unused lexical variables.
It's able to report access to unbound symbols in a large
number of code patterns.  That helps detect a lot of coding mistakes
and that's very valuable.

It may be difficult, or perhaps even not possible, to prevent the warning
in the situation I reported.  If the byte compiler cannot be improve to
handle
this situation, could this scenario be added to the list of know limitations
of the byte-compiler?  Maybe someday it will become possible to handle it
and this scenario will help the process?


-- 
/Pierre

[-- Attachment #2: Type: text/html, Size: 2427 bytes --]

bug#49749: 26.3; 26.3 & 27.2: invalid byte compiler warning in short-circuited or form

[Michael Heerdegen]

Pierre Rouleau <prouleau001@gmail.com> writes:

> On Wed, Jul 28, 2021 at 11:26 AM Lars Ingebrigtsen <larsi@gnus.org> wrote:
>
>  Pierre Rouleau <prouleau001@gmail.com> writes:
>
>  > (defun f-or ()
>  >   "Use or."
>  >   (when (or (null (boundp 'foo))
>  >             (null foo))  ;=> ``Warning: reference to free variable ‘foo’``                         
>  >     (message "foo is not set")))

> Well, from the perspective of a user, that would look at the 
> very least as a technical limitation.

There are a lot of possible expressions of the form

 (some-expression-involving-both (boundp 'foo) foo)

You have showed two.  There is an unlimited number of such expressions,
and it's impossible to solve the problem for all of them.  It's not hard
to support more, but for what gain?  The warning is still correct -
there _is_ a reference to a free variable.  It doesn't say "unbound".

When your reference to a free variable is not totally trivial, the
common method is to add a

  (defvar foo)

to your code.  That tells you and other readers and the compiler that
`foo' is a special variable defined elsewhere, and the warning is also
gone.

Also, (and (boundp 'foo) foo) is equivalent to (bound-and-true-p foo),
so there is a canonical form for that kind of test that doesn't lead to
a warning.  For more complex real-life situations, you will probably
need a `defvar' anyway, so we are really only discussing about
rewritings of `bound-and-true-p' using De Morgan rules.  I agree to Lars
that this would not be a useful thing to do.

Michael.

bug#49749: 26.3; 26.3 & 27.2: invalid byte compiler warning in short-circuited or form

[Lars Ingebrigtsen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> I agree to Lars that this would not be a useful thing to do.

So I'm closing this bug report.

(The issue isn't that the compiler warning is wrong -- it's correct; but
we suppress it in very particular situations where it'll obviously not
lead to any problems.)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no