bug#55447: 29.0.50; package-quickstart.el contains '#$', preventing Emacs from initializing correctly

bug#55447: 29.0.50; package-quickstart.el contains '#$', preventing Emacs from initializing correctly

[Daniel Semyonov via Bug reports for GNU Emacs, the Swiss army knife of text editors]

After invoking 'package-quickstart-refresh' (either manually or
automatically), 'package-quickstart.el' contains the following sexp for
each package:

(add-to-list 'load-path (directory-file-name
                         (or (file-name-directory #$) (car load-path))))
                                                  ^^

This produces the following error when trying to run Emacs graphically:

$ emacs
Wrong type argument: stringp, nil

No frame is shown afterwards.

When launching Emacs using 'emacs -nw' a frame is created, but the user
init file isn't executed, and the following line appears at the start of
the '*Messages*' buffer:

byte-code: Wrong type argument: stringp, nil

While I'm not sure what should be present instead of '#$', replacing
each occurrence with 'load-true-file-name' and recompiling the file
manually seems to fix the issue.

Recipe (not sure how useful it is in this case honestly):

1. $ emacs -Q
2. M-x package-quickstart-refresh RET


In GNU Emacs 29.0.50 (build 14, x86_64-pc-linux-gnu, GTK+ Version 3.24.33, cairo version 1.16.0)
 of 2022-05-16 built on coldharbour
Repository revision: 1f76f50a049bcbd43ff2c0dbe1e73b6fa5efc8b5
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12101003
System Description: Debian GNU/Linux bookworm/sid

Configured using:
 'configure --with-x-toolkit=gtk3 --with-xinput2
 --with-native-compilation --with-xwidgets --prefix=/opt/Emacs'

Configured features:
CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG JSON
LCMS2 LIBSELINUX LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG
RSVG SECCOMP SOUND SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS WEBP X11
XDBE XIM XINPUT2 XPM XWIDGETS GTK3 ZLIB

Important settings:
  value of $LC_CTYPE: en_US.UTF-8
  value of $LANG: en_IL
  locale-coding-system: utf-8-unix

Major mode: Group

Minor modes in effect:
  gnus-topic-mode: t
  init-gnus-group-key-binding-mode: t
  gnus-undo-mode: t
  emms-mode-line-mode: t
  emms-playing-time-display-mode: t
  emms-playing-time-mode: t
  electric-pair-mode: t
  delete-selection-mode: t
  pixel-scroll-precision-mode: t
  repeat-mode: t
  vcomplete-mode: t
  display-time-mode: t
  swsw-mode: t
  global-auto-revert-mode: t
  minibuffer-electric-default-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tab-bar-mode: t
  file-name-shadow-mode: t
  context-menu-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  buffer-read-only: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(gnus-topic nndraft nnmh nnml nnfolder utf-7 epa-file init-gnus
gnus-demon xref dabbrev vc vc-git diff-mode vc-dispatcher shadow
emacsbug flow-fill sort smiley gnus-cite qp mail-extr textsec
uni-scripts idna-mapping ucs-normalize uni-confusable textsec-check
gnus-async gnus-bcklg gnus-agent gnus-srvr gnus-score score-mode
nnvirtual nntp gnus-ml gnus-msg disp-table nndoc gnus-cache gnus-dup
gnus-art mm-uu mml2015 mm-view mml-smime smime dig gnus-sum shr
pixel-fill kinsoku url-file url-dired svg dom gnus-group gnus-undo
gnus-start gnus-dbus gnus-cloud nnimap nnmail mail-source utf7 netrc
nnoo parse-time iso8601 gnus-spec gnus-int gnus-range gnus-win gnus
nnheader range misearch multi-isearch mm-archive message sendmail
yank-media rfc822 mml mml-sec epa derived epg rfc6068 epg-config
gnus-util mailabbrev gmm-utils mailheader mail-utils gnutls
network-stream url-cache crm debbugs-gnu add-log debbugs-compat debbugs
soap-client mm-decode mm-bodies mm-encode url-http url-auth mail-parse
rfc2231 rfc2047 rfc2045 mm-util ietf-drums mail-prsvr url-gw nsm rmc
puny rng-xsd rng-dt rng-util xsd-regexp comp comp-cstr rx noutline
outline easy-mmode jka-compr find-func shortdoc package-x ffap cl-extra
help-fns radix-tree cus-start time-date checkdoc lisp-mnt help-mode
flymake-proc flymake project compile text-property-search comint
ansi-color warnings thingatpt flyspell ispell display-line-numbers
modus-operandi-theme cal-julian modus-themes solar cal-dst cal-menu
calendar cal-loaddefs init-interfaces init-web init-volume init-emms
emms-mode-line emms-player-mpv emms-playing-time emms-player-simple
emms-info-opusinfo emms-info emms-later-do emms emms-compat init-battery
battery dbus xml init-programs dired dired-loaddefs init-shell init-pass
auth-source-pass init-programming elec-pair init-ui delsel pixel-scroll
cua-base ring format-spec repeat vcomplete time swsw autorevert
filenotify minibuf-eldef init-personal pcase init-base server finder-inf
package browse-url url url-proxy url-privacy url-expand url-methods
url-history url-cookie url-domsuf url-util mailcap url-handlers
url-parse auth-source cl-seq eieio eieio-core cl-macs eieio-loaddefs
password-cache json map url-vars seq gv subr-x byte-opt bytecomp
byte-compile cconv info cus-edit pp cus-load wid-edit cl-loaddefs cl-lib
iso-transl tooltip eldoc paren electric uniquify ediff-hook vc-hooks
lisp-float-type elisp-mode mwheel term/x-win x-win term/common-win x-dnd
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace
newcomment text-mode lisp-mode prog-mode register page tab-bar menu-bar
rfn-eshadow isearch easymenu timer select scroll-bar mouse jit-lock
font-lock syntax font-core term/tty-colors frame minibuffer nadvice
simple cl-generic cham georgian utf-8-lang misc-lang vietnamese tibetan
thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian
slovak czech european ethiopic indian cyrillic chinese composite
emoji-zwj charscript charprop case-table epa-hook jka-cmpr-hook help
abbrev obarray oclosure cl-preloaded button loaddefs faces cus-face
macroexp files window text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget keymap hashtable-print-readable backquote
threads xwidget-internal dbusbind inotify lcms2 dynamic-setting
system-font-setting font-render-setting cairo move-toolbar gtk x-toolkit
xinput2 x multi-tty make-network-process native-compile emacs)

Memory information:
((conses 16 773902 108932)
 (symbols 48 27617 10)
 (strings 32 141862 12766)
 (string-bytes 1 5102638)
 (vectors 16 83777)
 (vector-slots 8 2043163 91780)
 (floats 8 3262 477)
 (intervals 56 2521 1188)
 (buffers 992 32))

bug#55447: 29.0.50; package-quickstart.el contains '#$', preventing Emacs from initializing correctly

[Lars Ingebrigtsen]

Daniel Semyonov <daniel@dsemy.com> writes:

> After invoking 'package-quickstart-refresh' (either manually or
> automatically), 'package-quickstart.el' contains the following sexp for
> each package:
>
> (add-to-list 'load-path (directory-file-name
>                          (or (file-name-directory #$) (car load-path))))
>                                                   ^^

I suspect that you have some .elc files that have been corrupted by an
earlier Emacs version at some point (I vaguely remember there being an
error here for a couple of days a while ago).  Try deleting all the .elc
files in your package directory and see whether that helps.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#55447: 29.0.50; package-quickstart.el contains '#$', preventing Emacs from initializing correctly

[Daniel Semyonov via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>>>>> Lars Ingebrigtsen writes:

    > Try deleting all the .elc files in your package directory and see
    > whether that helps.

I have deleted 'package-quickstart.el{,c}', all '*.elc' files in my
package directory, restarted Emacs, then regenerated
'package-quickstart.el', and the same issue occurs.

BTW, initially I assumed this was an issue with my configuration
somehow, but it now seems that two more people have encountered this
issue (https://lists.gnu.org/archive/html/emacs-devel/2022-05/msg00702.html).

Thanks for looking into this.

bug#55447: 29.0.50; package-quickstart.el contains '#$', preventing Emacs from initializing correctly

[Lars Ingebrigtsen]

Daniel Semyonov <daniel@dsemy.com> writes:

> I have deleted 'package-quickstart.el{,c}', all '*.elc' files in my
> package directory, restarted Emacs, then regenerated
> 'package-quickstart.el', and the same issue occurs.

I can reproduce this problem, too, but so far, I don't understand it.
The quickstart file includes these things from the package autoloads:

---
(add-to-list 'load-path (directory-file-name
                         (or (file-name-directory #$) (car load-path))))
---

Which...  is wrong on many levels, I think?

Because this would just put ~/.emacs.d/ into the load path if it worked
(which isn't what we wanted to achieve), and the `or' statement itself
is slightly nonsensical (what is it really trying to test here? #$
should always return a file name?)

But the regression is that #$ returns nil now in this case, and I'm not
sure why.  Anybody?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#55447: 29.0.50; package-quickstart.el contains '#$', preventing Emacs from initializing correctly

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Which...  is wrong on many levels, I think?

D'oh, I missed this bit:

(let ((load-true-file-name "/home/larsi/.emacs.d/elpa/xelb-0.18/xelb-autoloads.el")(load-file-name "/home/larsi/.emacs.d/elpa/xelb-0.18/xelb-autoloads.el"))

(add-to-list 'load-path (directory-file-name
                         (or (file-name-directory #$) (car load-path))))

So that's OK, then.

It seems like the package-quickstart.elc file is broken...  hm...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#55447: 29.0.50; package-quickstart.el contains '#$', preventing Emacs from initializing correctly

[Lars Ingebrigtsen]

In Emacs 28, the quickstart file has:

(add-to-list 'load-path (directory-file-name
                         (or (file-name-directory "/home/larsi/.emacs.d/elpa/xelb-0.18/xelb-autoloads.el") (car load-path))))

So it's not expanding those #$ things any more when generating
autoloads?  Is this something that started happening in the last few
days?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#55447: 29.0.50; package-quickstart.el contains '#$', preventing Emacs from initializing correctly

[Lars Ingebrigtsen]

The problem is here:

          ;; Fixup the special #$ reader form and throw away comments.
          (while (re-search-forward "#\\$\\|^;\\(.*\n\\)" nil 'move)
            (unless (nth 8 (syntax-ppss))
              (replace-match (if (match-end 1) "" pfile) t t)))

This assumes that syntax-ppss doesn't change match data, and that was
apparently true in Emacs 28, but is no longer true.  Slapping a
(save-match-data ...) around that fixes this problem.

Stefan, is syntax-ppss supposed to not clobber match data?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#55447: 29.0.50; package-quickstart.el contains '#$', preventing Emacs from initializing correctly

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Lars Ingebrigtsen [2022-05-17 11:46:18] wrote:
> Stefan, is syntax-ppss supposed to not clobber match data?

No, that's an unsafe assumption.
Thanks for spotting the problem,


        Stefan

bug#55447: 29.0.50; package-quickstart.el contains '#$', preventing Emacs from initializing correctly

[Lars Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Stefan, is syntax-ppss supposed to not clobber match data?
>
> No, that's an unsafe assumption.

Thanks; then my changes to `package-quickstart-refresh' in Emacs 29
should be sufficient to fix the problem, and I'm therefore closing this
bug report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no