bug#67321: 29.1.90; Different parsing rules for -*- lexical-binding:t; -*- in different circumstances

bug#67321: 29.1.90; Different parsing rules for -*- lexical-binding:t; -*- in different circumstances

[Sebastian Miele]

Put the following into a file named, e.g., ‘test-script’, and ‘chmod +x’
it.

  #!/bin/sh
  : ; exec emacs --script "$0" -- "$@" #; -*- lexical-binding: t; mode: emacs-lisp; -*-

  (defmacro lexical-binding-p ()
    '(let* ((x t)
            (f (lambda () x))
            (x nil))
       (funcall f)))

  (message "%s %s" lexical-binding (lexical-binding-p))

When the script is run, the output is "nil nil", signifying that lexical
binding is not enabled.

Then find the file in an interactive Emacs session, and interactively
evaluate (C-x C-e) the two expressions.  The output now is "t t", i.e.,
lexical binding is in use.

https://lists.gnu.org/archive/html/emacs-devel/2023-11/msg01041.html
contains more details, which are repeated here:

> From: Jens Schmidt <jschmidt4gnu@vodafonemail.de>
> Date: Mon, 2023-11-20 21:10 +0100
>
> I tried byte-compiling something similar yesterday, which also
> indicated that the byte-compiler compiles with lexical bindings.  Only
> the scripting machinery sees dynamical bindings.
>
> […]  It seems that the scripting machinery expects a semicolon in the
> very first column, without that the lexical-binding line is not
> recognized.  Even a space before the semicolon breaks the recognition.
>
> The problem is in function `lisp_file_lexically_bound_p' from lread.c,
> which is indeed much more strict in its recognition of the -*- ... -*-
> stanza than the functions `set-auto-mode-1' and
> `hack-local-variables-prop-line' from files.el.  The Emacs manual
> ((emacs) Specifying File Variables) only mentions that the stanza has
> to be in the first line (or the second one if the first is taken by a
> she-bang), without any restriction where the comment has to start.

((emacs) Specifying File Variables) states no restrictions on what must
(not) precede or follow a "-*- … -*-" on the first or second line of a
file.

Expected: At least consistent behavior.  Ideally, lexical binding
should be enabled in all cases.

In GNU Emacs 29.1.90 (build 1, x86_64-pc-linux-gnu, GTK+ Version
 3.24.38, cairo version 1.18.0) of 2023-11-17 built on blatt
Repository revision: b759f9b508da20649a0741bcfad466e6111edb4e
Repository branch: pgtk-hyper
System Description: Arch Linux

Configured using:
 'configure --with-pgtk --with-native-compilation --with-tree-sitter
 --with-json --with-mailutils --with-imagemagick
 'CFLAGS=-march=ivybridge -mtune=ivybridge''

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ
IMAGEMAGICK JPEG JSON LCMS2 LIBOTF LIBSYSTEMD LIBXML2 MODULES
NATIVE_COMP NOTIFY INOTIFY PDUMPER PGTK PNG RSVG SECCOMP SOUND SQLITE3
THREADS TIFF TOOLKIT_SCROLL_BARS TREE_SITTER WEBP XIM GTK3 ZLIB

Important settings:
  value of $LANG: C.UTF-8
  locale-coding-system: utf-8-unix

Major mode: mu4e:view

Minor modes in effect:
  mu4e-search-minor-mode: t
  mu4e-context-minor-mode: t
  envrc-global-mode: t
  envrc-mode: t
  editorconfig-mode: t
  global-git-commit-mode: t
  magit-auto-revert-mode: t
  marginalia-mode: t
  corfu-popupinfo-mode: t
  corfu-history-mode: t
  corfu-candidate-overlay-mode: t
  global-corfu-mode: t
  corfu-mode: t
  info-rename-buffer-mode: t
  savehist-mode: t
  server-mode: t
  recentf-mode: t
  vertico-mode: t
  shell-dirtrack-mode: t
  minibuffer-depth-indicate-mode: t
  global-so-long-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  buffer-read-only: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  temp-buffer-resize-mode: t

Load-path shadows:
/home/w/dot/emacs/.init.d/usr/elpa/modus-themes-20231115.1302/theme-loaddefs hides /home/w/usr/emacs/0/29/0/lisp/theme-loaddefs
/home/w/dot/emacs/.init.d/usr/elpa/transient-20231112.923/transient hides /home/w/usr/emacs/0/29/0/lisp/transient

Features:
(shadow emacsbug sort smiley gnus-cite mail-extr textsec uni-scripts
idna-mapping ucs-normalize uni-confusable textsec-check macrostep-c
cmacexp make-mode tabify tramp-cmds misearch multi-isearch goto-addr
vc-hg vc-bzr vc-src vc-sccs vc-svn vc-cvs vc-rcs log-view
emacsql-sqlite-builtin sqlite face-remap cal-move oc-basic ol-eww
ol-rmail ol-mhe ol-irc ol-info ol-gnus nnselect ol-docview doc-view
jka-compr image-mode exif ol-bibtex bibtex ol-bbdb ol-w3m ol-doi
org-link-doi cus-start whx-init-later whx-tangled-mu mu4e mu4e-org
mu4e-notification notifications mu4e-main mu4e-view gnus-art mm-uu
mml2015 mm-view mml-smime smime dig gnus-sum gnus-group gnus-undo
gnus-start gnus-dbus dbus gnus-cloud nnimap nnmail mail-source utf7 nnoo
gnus-spec gnus-int gnus-range gnus-win mu4e-headers mu4e-compose
mu4e-draft mu4e-actions smtpmail mu4e-search mu4e-lists mu4e-bookmarks
mu4e-mark mu4e-message flow-fill mule-util hl-line mu4e-contacts
mu4e-update mu4e-folders mu4e-context mu4e-query-items mu4e-server
mu4e-modeline mu4e-vars mu4e-helpers mu4e-config mu4e-window ido
mu4e-obsolete whx-tangled-envrc eglot external-completion array jsonrpc
ert ewoc flymake-proc flymake envrc inheritenv whx-tangled-org ob-python
ob-shell whx-custom org-superstar ol-eshell ol-man whx-password
whx-pcase whx-walk ox-odt rng-loc rng-uri rng-parse rng-match rng-dt
rng-util rng-pttrn nxml-parse nxml-ns nxml-enc xmltok nxml-util ox-latex
ox-icalendar org-agenda ox-html table ox-ascii ox-publish ox whx-etc-org
whx-tangled-0 web-mode advice disp-table css-mode sgml-mode facemenu eww
url-queue shr pixel-fill kinsoku url-file mm-url gnus nnheader range
docker-compose-mode dockerfile-mode apache-mode systemd cmake-font-lock
cmake-mode rst python sh-script smie executable lua-mode json-mode
json-snatcher js c-ts-common cc-mode cc-fonts cc-guess cc-menus cc-cmds
cc-styles cc-align cc-engine yaml-mode vcard tex-mode compile
consult-register vc-git consult-xref dumb-jump popup xref project
gptel-transient gptel gptel-openai editorconfig editorconfig-core
editorconfig-core-handle editorconfig-fnmatch ace-window avy wdired
dired-filter dired-hacks-utils dired-aux dired-x diary-lib
diary-loaddefs cal-iso forge-list forge-commands forge-semi
forge-bitbucket buck forge-gogs gogs forge-gitea gtea forge-gitlab glab
forge-github ghub-graphql treepy gsexp ghub url-http url-gw nsm url-auth
let-alist gnutls forge-notify forge-revnote forge-pullreq forge-issue
forge-topic yaml bug-reference forge-post markdown-mode forge-repo forge
forge-core forge-db closql emacsql-sqlite-common emacsql
emacsql-compiler eieio-base magit-patch magit-extras magit-bookmark
magit-submodule magit-blame magit-stash magit-reflog magit-bisect
magit-push magit-pull magit-fetch magit-clone magit-remote magit-commit
magit-sequence magit-notes magit-worktree magit-tag magit-merge
magit-branch magit-reset magit-files magit-refs magit-status magit
magit-repos magit-apply magit-wip magit-log which-func magit-diff
smerge-mode diff diff-mode easy-mmode git-commit log-edit message
sendmail yank-media puny rfc822 mml mml-sec epa derived epg rfc6068
epg-config gnus-util mm-decode mm-bodies mm-encode mail-parse rfc2231
rfc2047 rfc2045 mm-util ietf-drums mail-prsvr mailabbrev mail-utils
gmm-utils mailheader pcvs-util add-log magit-core magit-autorevert
autorevert filenotify magit-margin magit-transient magit-process
with-editor magit-mode magit-git magit-base magit-section cursor-sensor
git-modes gitignore-mode gitconfig-mode conf-mode gitattributes-mode
em-hist em-pred esh-mode eshell esh-cmd esh-ext esh-opt esh-proc esh-io
esh-arg esh-module esh-groups esh-util marginalia vertico-quick
corfu-popupinfo corfu-info kind-icon svg-lib color svg dom xml
corfu-history corfu-quick corfu-candidate-overlay corfu orderless crm
man info-rename-buffer helpful cc-langs cc-vars cc-defs imenu trace
edebug info-look f f-shortdoc elisp-refs help-fns radix-tree embark-org
org-element org-persist xdg org-id org-refile avl-tree generator
embark-consult embark ffap thingatpt ispell savehist tramp-cache
time-stamp expreg treesit cap-words superword subword rect whx-cl server
consult-dir consult recentf tree-widget bookmark text-property-search
vertico-directory vertico tramp-sh tramp-rclone tramp-fuse tramp
tramp-loaddefs trampver tramp-integration files-x tramp-compat shell
parse-time iso8601 mb-depth whx-safe-local whx-etc-2 whx-transient comp
comp-cstr warnings whx-arg whx-meta transient cl-extra edmacro kmacro
compat whx-etc-elisp whx-bind whx-elisp whx-feature whx-string s
whx-error debug backtrace help-mode macrostep whx-etc-look dired
dired-loaddefs org ob ob-tangle ob-ref ob-lob ob-table ob-exp org-macro
org-src ob-comint org-pcomplete pcomplete comint ansi-osc ansi-color
ring org-list org-footnote org-faces org-entities time-date noutline
outline ob-emacs-lisp ob-core ob-eval org-cycle org-table ol org-fold
org-fold-core org-keys oc org-loaddefs find-func cal-menu calendar
cal-loaddefs org-version org-compat org-macs format-spec whx-face
whx-alist whx-plist whx-list dash modus-operandi-theme modus-themes
diminish whx-etc-packages--core finder-inf ace-window-autoloads
apache-mode-autoloads async-autoloads avy-autoloads
cmake-font-lock-autoloads cmake-mode-autoloads consult-dir-autoloads
consult-project-extra-autoloads corfu-candidate-overlay-autoloads
corfu-autoloads debbugs-autoloads diminish-autoloads
dired-collapse-autoloads dired-filter-autoloads dired-subtree-autoloads
dired-hacks-utils-autoloads docker-compose-mode-autoloads
dockerfile-mode-autoloads dumb-jump-autoloads editorconfig-autoloads
eglot-tempel-autoloads embark-consult-autoloads consult-autoloads
embark-autoloads envrc-autoloads expand-region-autoloads
expreg-autoloads forge-autoloads closql-autoloads emacsql-autoloads
geiser-guile-autoloads geiser-autoloads ghub-autoloads
git-modes-autoloads gptel-autoloads helpful-autoloads
elisp-refs-autoloads f-autoloads i3wm-config-mode-autoloads
info-rename-buffer-autoloads inheritenv-autoloads json-mode-autoloads
json-snatcher-autoloads kbd-mode-autoloads keycast-autoloads
kind-icon-autoloads lua-mode-autoloads macrostep-autoloads
magit-autoloads git-commit-autoloads magit-section-autoloads
dash-autoloads marginalia-autoloads markdown-mode-autoloads
modus-themes-autoloads mu4e-autoloads orderless-autoloads
org-superstar-autoloads pkgbuild-mode-autoloads popup-autoloads
quelpa-autoloads rainbow-mode-autoloads s-autoloads svg-lib-autoloads
systemd-autoloads tempel-collection-autoloads tempel-autoloads
transient-autoloads treepy-autoloads vcard-autoloads vertico-autoloads
vimrc-mode-autoloads web-mode-autoloads wgrep-autoloads
with-editor-autoloads info compat-autoloads yaml-autoloads
yaml-mode-autoloads whx-package--core package-vc vc vc-dispatcher
lisp-mnt package browse-url url url-proxy url-privacy url-expand
url-methods url-history url-cookie generate-lisp-file url-domsuf
url-util mailcap url-handlers url-parse auth-source eieio eieio-core
password-cache json map byte-opt bytecomp byte-compile url-vars
whx-etc-1--core so-long whx-etc-buffer--core ibuffer ibuffer-loaddefs
whx-etc-elisp--core whx-elisp--core rx whx-etc-gc--core whx-bind--core
whx-init-early whx-boot whx-etc-0--core cus-edit pp cus-load icons
wid-edit whx-etc-native-comp--core whx-etc-litter--core no-littering
whx-etc-fs-path--core whx-fs-path--core cl-seq whx-string--core
whx-log--core whx-0--core whx-let-set--core whx-symbol--core
whx-list--core subr-x whx-meta--core whx-pcase--core pcase
whx-error--core whx-binary-unit-prefix--core whx-feature--core cl-macs
gv cl-loaddefs cl-lib rmc iso-transl tooltip cconv eldoc paren electric
uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel
term/pgtk-win pgtk-win term/common-win pgtk-dnd tool-bar dnd fontset
image regexp-opt fringe tabulated-list replace newcomment text-mode
lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch
easymenu timer select scroll-bar mouse jit-lock font-lock syntax
font-core term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads dbusbind inotify
dynamic-setting system-font-setting font-render-setting cairo gtk pgtk
lcms2 multi-tty make-network-process native-compile emacs)

Memory information:
((conses 16 1580890 49378)
 (symbols 48 74132 9)
 (strings 32 392309 15713)
 (string-bytes 1 11053956)
 (vectors 16 155349)
 (vector-slots 8 3761081 202439)
 (floats 8 807 1807)
 (intervals 56 61711 3388)
 (buffers 984 46))

bug#67321: 29.1.90; Different parsing rules for -*- lexical-binding:t; -*- in different circumstances

[Andreas Schwab]

There are no different parsing rules.  emacs --script just doesn't pay
attention to file local variables.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

bug#67321: 29.1.90; Different parsing rules for -*- lexical-binding:t; -*- in different circumstances

[Gerd Möllmann]

Sebastian Miele <iota@whxvd.name> writes:

> Put the following into a file named, e.g., ‘test-script’, and ‘chmod +x’
> it.
>
>   #!/bin/sh
>   : ; exec emacs --script "$0" -- "$@" #; -*- lexical-binding: t; mode: emacs-lisp; -*-
>
>   (defmacro lexical-binding-p ()
>     '(let* ((x t)
>             (f (lambda () x))
>             (x nil))
>        (funcall f)))
>
>   (message "%s %s" lexical-binding (lexical-binding-p))
>
> When the script is run, the output is "nil nil", signifying that lexical
> binding is not enabled.
>
> Then find the file in an interactive Emacs session, and interactively
> evaluate (C-x C-e) the two expressions.  The output now is "t t", i.e.,
> lexical binding is in use.
>
> https://lists.gnu.org/archive/html/emacs-devel/2023-11/msg01041.html
> contains more details, which are repeated here:
>
>> From: Jens Schmidt <jschmidt4gnu@vodafonemail.de>
>> Date: Mon, 2023-11-20 21:10 +0100
>>
>> I tried byte-compiling something similar yesterday, which also
>> indicated that the byte-compiler compiles with lexical bindings.  Only
>> the scripting machinery sees dynamical bindings.
>>
>> […]  It seems that the scripting machinery expects a semicolon in the
>> very first column, without that the lexical-binding line is not
>> recognized.  Even a space before the semicolon breaks the recognition.
>>
>> The problem is in function `lisp_file_lexically_bound_p' from lread.c,
>> which is indeed much more strict in its recognition of the -*- ... -*-
>> stanza than the functions `set-auto-mode-1' and
>> `hack-local-variables-prop-line' from files.el.  The Emacs manual
>> ((emacs) Specifying File Variables) only mentions that the stanza has
>> to be in the first line (or the second one if the first is taken by a
>> she-bang), without any restriction where the comment has to start.
>
> ((emacs) Specifying File Variables) states no restrictions on what must
> (not) precede or follow a "-*- … -*-" on the first or second line of a
> file.
>
> Expected: At least consistent behavior.  Ideally, lexical binding
> should be enabled in all cases.
>
> In GNU Emacs 29.1.90 (build 1, x86_64-pc-linux-gnu, GTK+ Version

Same in master, BTW, and everything you write is correct, I think.

I guess one could recognize the ": ; exec" idiom (do nothing
successfully and then exec emacs) in command-line--load-script, but that
feels really strange and brittle.

Can I ask why that idiom is used?

bug#67321: 29.1.90; Different parsing rules for -*- lexical-binding:t; -*- in different circumstances

[Sebastian Miele]

> From: Andreas Schwab <schwab@suse.de>
> Date: Tue, 2023-11-21 11:06 +0100
>
> There are no different parsing rules.  emacs --script just doesn't pay
> attention to file local variables.

That cannot be true.  Running

  #!/path/to/emacs --script
  ;; -*- lexical-binding: t; mode: emacs-lisp; -*-
  (defmacro lexical-binding-p ()
    '(let* ((x t)
            (f (lambda () x))
            (x nil))
       (funcall f)))
  (message "%s %s" lexical-binding (lexical-binding-p))

as a script outputs "t t".  When I change the "lexical-binding: t" into
"lexical-binding: nil", the output is "nil nil".  So that special
file-local variable does receive some handling by emacs --script.

bug#67321: 29.1.90; Different parsing rules for -*- lexical-binding:t; -*- in different circumstances

[Eli Zaretskii]

merge 67321 64272
thanks

> From: Sebastian Miele <iota@whxvd.name>
> Date: Tue, 21 Nov 2023 09:16:13 +0100
> 
> Put the following into a file named, e.g., ‘test-script’, and ‘chmod +x’
> it.
> 
>   #!/bin/sh
>   : ; exec emacs --script "$0" -- "$@" #; -*- lexical-binding: t; mode: emacs-lisp; -*-
> 
>   (defmacro lexical-binding-p ()
>     '(let* ((x t)
>             (f (lambda () x))
>             (x nil))
>        (funcall f)))
> 
>   (message "%s %s" lexical-binding (lexical-binding-p))
> 
> When the script is run, the output is "nil nil", signifying that lexical
> binding is not enabled.
> 
> Then find the file in an interactive Emacs session, and interactively
> evaluate (C-x C-e) the two expressions.  The output now is "t t", i.e.,
> lexical binding is in use.
> 
> https://lists.gnu.org/archive/html/emacs-devel/2023-11/msg01041.html
> contains more details, which are repeated here:
> 
> > From: Jens Schmidt <jschmidt4gnu@vodafonemail.de>
> > Date: Mon, 2023-11-20 21:10 +0100
> >
> > I tried byte-compiling something similar yesterday, which also
> > indicated that the byte-compiler compiles with lexical bindings.  Only
> > the scripting machinery sees dynamical bindings.
> >
> > […]  It seems that the scripting machinery expects a semicolon in the
> > very first column, without that the lexical-binding line is not
> > recognized.  Even a space before the semicolon breaks the recognition.
> >
> > The problem is in function `lisp_file_lexically_bound_p' from lread.c,
> > which is indeed much more strict in its recognition of the -*- ... -*-
> > stanza than the functions `set-auto-mode-1' and
> > `hack-local-variables-prop-line' from files.el.  The Emacs manual
> > ((emacs) Specifying File Variables) only mentions that the stanza has
> > to be in the first line (or the second one if the first is taken by a
> > she-bang), without any restriction where the comment has to start.
> 
> ((emacs) Specifying File Variables) states no restrictions on what must
> (not) precede or follow a "-*- … -*-" on the first or second line of a
> file.
> 
> Expected: At least consistent behavior.  Ideally, lexical binding
> should be enabled in all cases.

There's more here than meets the eye, see bug#64272.

bug#67321: 29.1.90; Different parsing rules for -*- lexical-binding:t; -*- in different circumstances

[Sebastian Miele]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Date: Tue, 2023-11-21 11:46 +0100
>
>>   #!/bin/sh
>>   : ; exec emacs --script "$0" -- "$@" #; -*- lexical-binding: t; mode: emacs-lisp; -*-
>>
>>   (defmacro lexical-binding-p ()
>>     '(let* ((x t)
>>             (f (lambda () x))
>>             (x nil))
>>        (funcall f)))
>>
>>   (message "%s %s" lexical-binding (lexical-binding-p))
>
> Can I ask why that idiom is used?

The idiom allows arbitrary shell processing before, and possibly even
after, actually running Emacs in the script.

See https://github.com/doomemacs/doomemacs/blob/master/bin/doom for an
example including both pre- and postprocessing, and lots of comments
about what is done there, and why.

Another case that recently popped up on emacs-devel is that it allows to
insert the "--" for cleanly separating the script commandline options
from options that Emacs may interpret, see
https://lists.gnu.org/archive/html/emacs-devel/2023-11/msg00896.html.

I do not know anymore why I considered the trick in the past.  I do not
want to spend the time to look into my scripts to find that bits that
would become cumbersome without such tricks.  Because of the problem
with lexical-binding not being picked up, and because I, for now, have
the luxury of only using Linux, I can use another trick.  But being able
to mangle the command line of Emacs before running Emacs definitely is a
useful thing.  The portable alternative would be to have one separate
(shell) wrapper around every Emacs script that needs such mangling.

What follows is the beginning of
https://github.com/doomemacs/doomemacs/blob/master/bin/doom for those
who do not visit GitHub:

#!/usr/bin/env sh
:; # -*- mode: emacs-lisp; lexical-binding: t -*-
:; case "$EMACS" in *term*) EMACS=emacs ;; *) EMACS="${EMACS:-emacs}" ;; esac
:; emacs="$EMACS ${DEBUG:+--debug-init} -q --no-site-file --batch"
:; tmpdir=`$emacs --eval '(princ (temporary-file-directory))' 2>/dev/null`
:; [ -z "$tmpdir" ] && { >&2 echo "Error: failed to run Emacs with command '$EMACS'"; >&2 echo; >&2 echo "Are you sure Emacs is installed and in your \$PATH?"; exit 1; }
:; export __DOOMPID="${__DOOMPID:-$$}"
:; export __DOOMSTEP="${__DOOMSTEP:-0}"
:; export __DOOMGEOM="${__DOOMGEOM:-`tput cols 2>/dev/null`x`tput lines 2>/dev/null`}"
:; export __DOOMGPIPE=${__DOOMGPIPE:-$__DOOMPIPE}
:; export __DOOMPIPE=; [ -t 0 ] || __DOOMPIPE="${__DOOMPIPE}0"; [ -t 1 ] || __DOOMPIPE="${__DOOMPIPE}1"
:; $emacs --load "$0" -- "$@" || exit=$?
:; [ "${exit:-0}" -eq 254 ] && { sh "${tmpdir}/doom.${__DOOMPID}.${__DOOMSTEP}.sh" "$0" "$@" && true; exit="$?"; }
:; exit $exit

;; This magical mess of a shebang is necessary for any script that relies on
;; Doom's CLI framework, because Emacs' tty libraries and capabilities are too
;; immature (borderline non-existent) at the time of writing (28.1). This
;; shebang sets out to accomplish these three goals:
;;
;; 1. To produce a more helpful error if Emacs isn't installed or broken. It
;;    must do so without assuming whether $EMACS is a shell command (e.g. 'snap
;;    run emacs') or an absolute path (to an emacs executable). I've avoided
;;    'command -v $EMACS' for this reason.
;;
;; 2. To allow this Emacs session to "exit into" a child process (since Elisp
;;    lacks an analogue for exec system calls) by calling an auto-generated and
;;    self-destructing "exit script" if the parent Emacs process exits with code
;;    254. It takes care to prevent nested child instances from clobbering the
;;    exit script.
;;
;; 3. To expose some information about the terminal and session:
;;    - $__DOOMGEOM holds the dimensions of the terminal (W . H).
;;    - $__DOOMPIPE indicates whether the script has been piped (in and/or out).
;;    - $__DOOMGPIPE indicates whether one of this process' parent has been
;;      piped to/from.
;;    - $__DOOMPID is a unique identifier for the parent script, so
;;      child processes can identify which persistent data files (like logs) it
;;      has access to.
;;    - $__DOOMSTEP counts how many levels deep we are in the dream (appending
;;      this to the exit script's filename avoids child processes clobbering the
;;      same exit script and causing read errors).
;;    - $TMPDIR (or $TEMP and $TMP on Windows) aren't guaranteed to have values,
;;      and mktemp isn't available on all systems, but you know what is? Emacs!
;;      So I use it to print `temporary-file-directory'. And it seconds as a
;;      quick sanity check for Emacs' existence (for goal #1).
;;
;; Other weird facts about this shebang line:
;;
;; - The :; hack exploits properties of : and ; in shell scripting and elisp to
;;   allow shell script and elisp to coexist in the same file without either's
;;   interpreter throwing foreign syntax errors:
;;
;;   - In elisp, ":" is a valid keyword symbol literal; it evaluates to itself
;;     and has no side-effect.
;;   - In the shell, ":" is a valid command that does nothing and ignores its
;;     arguments.
;;   - In elisp, ";" begins a comment. I.e. the interpreter ignores everything
;;     after it.
;;   - In the shell, ";" is a command separator.
;;
;;   Put together, plus a strategically placed exit call, the shell will read
;;   one part of this file and ignore the rest, while the elisp interpreter will
;;   do the opposite.
;; - I intentionally avoid loading site files, so lisp/doom-cli.el can load them
;;   by hand later. There, I can suppress and deal with unhelpful warnings (e.g.
;;   "package cl is deprecated"), "Loading X...DONE" spam, and any other
;;   disasterous side-effects.
;;
;;   But be careful not to use -Q! It implies --no-site-lisp, which omits the
;;   site-lisp directory from `load-path'.
;; - POSIX-compliancy is paramount: there's no guarantee what /bin/sh will be
;;   symlinked to in the esoteric OSes/distros Emacs users use.
;; - The user may have a noexec flag set on /tmp, so pass the exit script to
;;   /bin/sh rather than executing them directly.

bug#67321: 29.1.90; Different parsing rules for -*- lexical-binding:t; -*- in different circumstances

[Gerd Möllmann]

Sebastian Miele <iota@whxvd.name> writes:

>> From: Gerd Möllmann <gerd.moellmann@gmail.com>
>> Date: Tue, 2023-11-21 11:46 +0100
>>
>>>   #!/bin/sh
>>>   : ; exec emacs --script "$0" -- "$@" #; -*- lexical-binding: t; mode: emacs-lisp; -*-
>>>
>>>   (defmacro lexical-binding-p ()
>>>     '(let* ((x t)
>>>             (f (lambda () x))
>>>             (x nil))
>>>        (funcall f)))
>>>
>>>   (message "%s %s" lexical-binding (lexical-binding-p))
>>
>> Can I ask why that idiom is used?
>
> The idiom allows arbitrary shell processing before, and possibly even
> after, actually running Emacs in the script.
>
> See https://github.com/doomemacs/doomemacs/blob/master/bin/doom for an
> example including both pre- and postprocessing, and lots of comments
> about what is done there, and why.
>
> Another case that recently popped up on emacs-devel is that it allows to
> insert the "--" for cleanly separating the script commandline options
> from options that Emacs may interpret, see
> https://lists.gnu.org/archive/html/emacs-devel/2023-11/msg00896.html.
>
> I do not know anymore why I considered the trick in the past.  I do not
> want to spend the time to look into my scripts to find that bits that
> would become cumbersome without such tricks.  Because of the problem
> with lexical-binding not being picked up, and because I, for now, have
> the luxury of only using Linux, I can use another trick.  But being able
> to mangle the command line of Emacs before running Emacs definitely is a
> useful thing.  The portable alternative would be to have one separate
> (shell) wrapper around every Emacs script that needs such mangling.

Thanks, Sebastian. I got the trick, though it took me a minute. Clever,
clever :-).

TBH, I don't see a good reason why that should be used instead of the
much simpler

cat >somefile <<EOF
; -*- lexical-binding: t -*-
(message "%s" lexical-binding)
EOF
emacs --script somefile

So, I personally would not like to spend the time to change Emacs,
sorry.

bug#67321: 29.1.90; Different parsing rules for -*- lexical-binding:t; -*- in different circumstances

[Sebastian Miele]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Date: Wed, 2023-11-22 15:59 +0100
>
> Thanks, Sebastian. I got the trick, though it took me a minute. Clever,
> clever :-).

I want to explicitly have mentioned that I did not come up with the
trick by myself.  (And it took me more than a minute to grok it.)

> TBH, I don't see a good reason why that should be used instead of the
> much simpler
>
> cat >somefile <<EOF
> ; -*- lexical-binding: t -*-
> (message "%s" lexical-binding)
> EOF
> emacs --script somefile

Didn't think of this either.  I'm not a big fan of the temporary file,
but it definitely is fine.

bug#67321: 29.1.90; Different parsing rules for -*- lexical-binding:t; -*- in different circumstances

[Gerd Möllmann]

Sebastian Miele <iota@whxvd.name> writes:

>> cat >somefile <<EOF
>> ; -*- lexical-binding: t -*-
>> (message "%s" lexical-binding)
>> EOF
>> emacs --script somefile
>
> Didn't think of this either.  I'm not a big fan of the temporary file,
> but it definitely is fine.

It's kind of a standard example of how to use here-documents.