bug#66628: 30.0.50; EMACS_GENERATE_DYNVARS emits symbols with position

bug#66628: 30.0.50; EMACS_GENERATE_DYNVARS emits symbols with position

[Basil L. Contovounesios via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 812 bytes --]

Severity: minor
Tags: patch

Given the following two files under /tmp:

  ;;; foo.el -*- lexical-binding: t -*-
  (defvar foo-x)
  (defvar foo-y)
  ;;; foo.el ends here

  ;;; bar.el -*- lexical-binding: t -*-
  (defun bar-x (foo-x) (ignore foo-x))
  (let ((foo-y 1))
    (ignore foo-y)
    (setq foo-y 2))
  ;;; bar.el ends here

  $ EMACS_GENERATE_DYNVARS=1 emacs -Q -batch -f batch-byte-compile foo.el
  Generating /tmp/foo.elc.dynvars

  $ cat foo.elc.dynvars
  (#<symbol foo-y at 62> . "/tmp/foo.el")
  (#<symbol foo-x at 47> . "/tmp/foo.el")

  $ EMACS_DYNVARS_FILE=foo.elc.dynvars emacs -Q -batch -f batch-byte-compile bar.el

Since the .dynvars file makes for some bad reading, the last step gives
no diagnostic, though I would expect it to.

Here's a patch that binds print-symbols-bare appropriately:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Print-bare-symbols-for-EMACS_GENERATE_DYNVARS.patch --]
[-- Type: text/x-diff, Size: 1519 bytes --]

From 492ad0f4a97ca43409eb0fefacdfe84b0775cf15 Mon Sep 17 00:00:00 2001
From: "Basil L. Contovounesios" <contovob@tcd.ie>
Date: Wed, 18 Oct 2023 15:33:51 +0200
Subject: [PATCH] Print bare symbols for EMACS_GENERATE_DYNVARS

* lisp/emacs-lisp/bytecomp.el (byte-compile--load-dynvars): Remove
redundant goto-char.
(byte-compile-file): Bind print-symbols-bare around .dynvars file
generation.
---
 lisp/emacs-lisp/bytecomp.el | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lisp/emacs-lisp/bytecomp.el b/lisp/emacs-lisp/bytecomp.el
index 92abe6b4624..9d97678379c 100644
--- a/lisp/emacs-lisp/bytecomp.el
+++ b/lisp/emacs-lisp/bytecomp.el
@@ -2069,7 +2069,6 @@ byte-compile--load-dynvars
   (and file (not (equal file ""))
        (with-temp-buffer
          (insert-file-contents file)
-         (goto-char (point-min))
          (let ((vars nil)
                var)
            (while (ignore-errors (setq var (read (current-buffer))))
@@ -2284,7 +2283,8 @@ byte-compile-file
         (let ((gen-dynvars (getenv "EMACS_GENERATE_DYNVARS")))
           (when (and gen-dynvars (not (equal gen-dynvars ""))
                      byte-compile--seen-defvars)
-            (let ((dynvar-file (concat target-file ".dynvars")))
+            (let ((dynvar-file (concat target-file ".dynvars"))
+                  (print-symbols-bare t))
               (message "Generating %s" dynvar-file)
               (with-temp-buffer
                 (dolist (var (delete-dups byte-compile--seen-defvars))
-- 
2.42.0


[-- Attachment #3: Type: text/plain, Size: 4324 bytes --]


Now I get the following, as expected:

  $ EMACS_GENERATE_DYNVARS=1 emacs -Q -batch -f batch-byte-compile foo.el
  Generating /tmp/foo.elc.dynvars

  $ cat foo.elc.dynvars
  (foo-y . "/tmp/foo.el")
  (foo-x . "/tmp/foo.el")

  $ EMACS_DYNVARS_FILE=foo.elc.dynvars emacs -Q -batch -f batch-byte-compile bar.el

  In bar-x:
  bar.el:2:2: Warning: ‘foo-x’ lexically bound in lambda here but declared dynamic in: /tmp/foo.el
  bar.el:3:8: Warning: ‘foo-y’ lexically bound in let here but declared dynamic in: /tmp/foo.el

WDYT?

Side question: if I replace

  (let ((foo-y 1))
    (ignore foo-y)
    (setq foo-y 2))

with

  (let ((foo-y 1))
    (ignore foo-y)
    (bar-x foo-y))

in bar.el (i.e. foo-y is only read, never set), the .dynvars warning
about foo-y disappears.  I assume this is due to an earlier cconv /
constant propagation pass, is that right?  If so, I'm curious
whether/how the .dynvars feature could be adapted to cope, e.g. by
taking place at an earlier stage.

Thanks,
-- 
Basil

In GNU Emacs 30.0.50 (build 1, x86_64-pc-linux-gnu, X toolkit, cairo
 version 1.18.0, Xaw3d scroll bars) of 2023-10-18 built on tia
Repository revision: 06fc5c24170b820939d3d51071b2957354edcb65
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12101008
System Description: Debian GNU/Linux trixie/sid

Configured using:
 'configure 'CFLAGS=-O2 -ggdb3' -C --prefix=/home/blc/.local
 --enable-checking=structs --with-file-notification=yes
 --with-x-toolkit=lucid --with-x'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES NOTIFY
INOTIFY PDUMPER PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF
TOOLKIT_SCROLL_BARS TREE_SITTER WEBP X11 XAW3D XDBE XIM XINPUT2 XPM
LUCID ZLIB

Important settings:
  value of $LANG: en_IE.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  minibuffer-regexp-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message mailcap yank-media puny dired
dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068
epg-config gnus-util text-property-search time-date subr-x mm-decode
mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
cl-loaddefs cl-lib sendmail rfc2047 rfc2045 ietf-drums mm-util
mail-prsvr mail-utils rmc iso-transl tooltip cconv eldoc paren electric
uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel
term/x-win x-win term/common-win x-dnd touch-screen tool-bar dnd fontset
image regexp-opt fringe tabulated-list replace newcomment text-mode
lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch
easymenu timer select scroll-bar mouse jit-lock font-lock syntax
font-core term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads dbusbind inotify lcms2
dynamic-setting system-font-setting font-render-setting cairo x-toolkit
xinput2 x multi-tty move-toolbar make-network-process emacs)

Memory information:
((conses 16 38082 9280) (symbols 48 5209 0) (strings 32 14036 1774)
 (string-bytes 1 387105) (vectors 16 10299)
 (vector-slots 8 156340 16868) (floats 8 23 23) (intervals 56 259 0)
 (buffers 992 10))

bug#66628: 30.0.50; EMACS_GENERATE_DYNVARS emits symbols with position

[Mattias Engdegård]

19 okt. 2023 kl. 08.32 skrev Basil L. Contovounesios <contovob@tcd.ie>:

> $ EMACS_GENERATE_DYNVARS=1 emacs -Q -batch -f batch-byte-compile foo.el
>  Generating /tmp/foo.elc.dynvars
> 
>  $ cat foo.elc.dynvars
>  (#<symbol foo-y at 62> . "/tmp/foo.el")
>  (#<symbol foo-x at 47> . "/tmp/foo.el")

Oops, good that you found this. The patch looks fine.
Thank you!

We could actually contemplate enriching the .dynvars format with source locations of the declarations when available, but it's perhaps just gold-plating at this point.

> Side question: if I replace
> 
>  (let ((foo-y 1))
>    (ignore foo-y)
>    (setq foo-y 2))
> 
> with
> 
>  (let ((foo-y 1))
>    (ignore foo-y)
>    (bar-x foo-y))
> 
> in bar.el (i.e. foo-y is only read, never set), the .dynvars warning
> about foo-y disappears.  I assume this is due to an earlier cconv /
> constant propagation pass, is that right?

Yes, constant propagation will substitute 1 for `foo-y` in the second case during Lisp-level optimisation because absent any dynamic declaration the variable is assumed to be lexical there. The warning is only emitted in the backend (codegen) which is admittedly a bit late.

>  If so, I'm curious
> whether/how the .dynvars feature could be adapted to cope, e.g. by
> taking place at an earlier stage.

Yes, it might be possible to move it to the frontend and in fact it's generally preferable to check for warnings at that stage. You could try doing that if you wish; I would like to make a frontend refactoring to make this sort of thing easier -- right now the frontend is part of cconv, except for the bit that's in macroexp-all.