bug#66022: 30.0.50; kmacro overwriting global keybindings

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Gerd Möllmann]

current master 1442f4043a761e9bdeeb4e1fbe9822c2987c1502, emacs -Q

  (keymap-global-set "S-<mouse-3>" 'ignore)

then C-x ( C-g.

  (keymap-lookup global-map "S-<mouse-3>")
  => kmacro-end-call-mouse

Notice that the binding has been overwritten.

This is actually not the binding I care about, but it's the only one I
can easily reproduce with emacs -Q.  With my init file, and with a
(trace-function 'global-set-key) I see

======================================================================
1 -> (global-set-key "(" kmacro-start-macro)
1 <- global-set-key: kmacro-start-macro
======================================================================
1 -> (global-set-key ")" kmacro-end-macro)
1 <- global-set-key: kmacro-end-macro
======================================================================
1 -> (global-set-key "e" kmacro-end-and-call-macro)
1 <- global-set-key: kmacro-end-and-call-macro
======================================================================
1 -> (global-set-key [f3] kmacro-start-macro-or-insert-counter)
1 <- global-set-key: kmacro-start-macro-or-insert-counter
======================================================================
1 -> (global-set-key [f4] kmacro-end-or-call-macro)
1 <- global-set-key: kmacro-end-or-call-macro
======================================================================
1 -> (global-set-key "\v" kmacro-keymap)
1 <- global-set-key: kmacro-keymap
======================================================================
1 -> (global-set-key "6" 2C-command)
1 <- global-set-key: 2C-command
======================================================================
1 -> (global-set-key [f2] 2C-command)
1 <- global-set-key: 2C-command

which overwrites part of my key bindings, also with ones from two-column.

(Note that two-column and kmacro are the only packages with autoloads
for global-set-key.  Maybe that plays a role, but TBH I don't have a
clue what's going on here.)

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Andreas Schwab]

On Sep 16 2023, Gerd Möllmann wrote:

> current master 1442f4043a761e9bdeeb4e1fbe9822c2987c1502, emacs -Q
>
>   (keymap-global-set "S-<mouse-3>" 'ignore)
>
> then C-x ( C-g.
>
>   (keymap-lookup global-map "S-<mouse-3>")
>   => kmacro-end-call-mouse
>
> Notice that the binding has been overwritten.

That's kmacro-call-mouse-event.

> This is actually not the binding I care about, but it's the only one I
> can easily reproduce with emacs -Q.  With my init file, and with a
> (trace-function 'global-set-key) I see
>
> ======================================================================
> 1 -> (global-set-key "(" kmacro-start-macro)
> 1 <- global-set-key: kmacro-start-macro
> ======================================================================
> 1 -> (global-set-key ")" kmacro-end-macro)
> 1 <- global-set-key: kmacro-end-macro
> ======================================================================
> 1 -> (global-set-key "e" kmacro-end-and-call-macro)
> 1 <- global-set-key: kmacro-end-and-call-macro
> ======================================================================
> 1 -> (global-set-key [f3] kmacro-start-macro-or-insert-counter)
> 1 <- global-set-key: kmacro-start-macro-or-insert-counter
> ======================================================================
> 1 -> (global-set-key [f4] kmacro-end-or-call-macro)
> 1 <- global-set-key: kmacro-end-or-call-macro
> ======================================================================
> 1 -> (global-set-key "\v" kmacro-keymap)
> 1 <- global-set-key: kmacro-keymap
> ======================================================================
> 1 -> (global-set-key "6" 2C-command)
> 1 <- global-set-key: 2C-command
> ======================================================================
> 1 -> (global-set-key [f2] 2C-command)
> 1 <- global-set-key: 2C-command
>
> which overwrites part of my key bindings, also with ones from two-column.

From kmacro.el:

;;; Provide some binding for startup:
;;;###autoload (global-set-key "\C-x(" #'kmacro-start-macro)
;;;###autoload (global-set-key "\C-x)" #'kmacro-end-macro)
;;;###autoload (global-set-key "\C-xe" #'kmacro-end-and-call-macro)
;;;###autoload (global-set-key [f3] #'kmacro-start-macro-or-insert-counter)
;;;###autoload (global-set-key [f4] #'kmacro-end-or-call-macro)
;;;###autoload (global-set-key "\C-x\C-k" #'kmacro-keymap)

They are all commented out (only to be put in loaddefs.el), so they
should not be executed when kmacro is loaded.

Maybe try make bootstrap?

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Gerd Möllmann]

Andreas Schwab <schwab@linux-m68k.org> writes:

> Maybe try make bootstrap?

It is already a clean build.

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Gerd Möllmann]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> Andreas Schwab <schwab@linux-m68k.org> writes:
>
>> Maybe try make bootstrap?
>
> It is already a clean build.

I now did this, before the C-x ( C-g in my recipe:

  (add-function :after (symbol-function 'global-set-key)
                (lambda (&rest args) (debug)))

and get the backtrace

Debugger entered: nil
  (closure (t) (&rest args) (debug))([S-mouse-3] kmacro-end-call-mouse)
  apply((closure (t) (&rest args) (debug)) ([S-mouse-3] kmacro-end-call-mouse))
  global-set-key([S-mouse-3] kmacro-end-call-mouse)
  byte-code("\302\300\10\"\210\11\203\20\0\303\304\11!\305\"\210\302\207" [kmacro-keymap kmacro-call-mouse-event defalias global-set-key vector kmacro-end-call-mouse] 3)
  autoload-do-load((autoload "kmacro" ("loaddefs.elc" . 753171) t nil) kmacro-start-macro)
  command-execute(kmacro-start-macro)

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Andreas Schwab]

As I already said, see kmacro-call-mouse-event.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Gerd Möllmann]

Andreas Schwab <schwab@linux-m68k.org> writes:

> As I already said, see kmacro-call-mouse-event.

(defcustom kmacro-call-mouse-event 'S-mouse-3
  "The mouse event used by kmacro to call a macro.
Set to nil if no mouse binding is desired."
  :type 'symbol)

You're being cryptic.

Care to elaborate?  You find it's not a bug, or a bug, can't be fixed,
don't want to fix?

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Andreas Schwab]

Follow the advice.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Gerd Möllmann]

Andreas Schwab <schwab@linux-m68k.org> writes:

> Follow the advice.

Could you please just tell what you know?

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Date: Sat, 16 Sep 2023 08:38:22 +0200
> 
> current master 1442f4043a761e9bdeeb4e1fbe9822c2987c1502, emacs -Q

The same problem exists on emacs-29 (and I presume in older versions
of Emacs).

>   (keymap-global-set "S-<mouse-3>" 'ignore)
> 
> then C-x ( C-g.

(The C-g part is not relevant; the same happens if you finish defining
the macro.)

>   (keymap-lookup global-map "S-<mouse-3>")
>   => kmacro-end-call-mouse
> 
> Notice that the binding has been overwritten.

This is because of this snippet at top level of kmacro.el:

  (if kmacro-call-mouse-event
    (global-set-key (vector kmacro-call-mouse-event) #'kmacro-end-call-mouse))

As the doc string of kmacro-call-mouse-event says:

  (defcustom kmacro-call-mouse-event 'S-mouse-3
    "The mouse event used by kmacro to call a macro.
  Set to nil if no mouse binding is desired."
    :type 'symbol)

customize this to nil to avoid overwriting the S-<mouse-3> binding you
have.

> This is actually not the binding I care about, but it's the only one I
> can easily reproduce with emacs -Q.  With my init file, and with a
> (trace-function 'global-set-key) I see
> 
> ======================================================================
> 1 -> (global-set-key "(" kmacro-start-macro)
> 1 <- global-set-key: kmacro-start-macro
> ======================================================================
> 1 -> (global-set-key ")" kmacro-end-macro)
> 1 <- global-set-key: kmacro-end-macro
> ======================================================================
> 1 -> (global-set-key "e" kmacro-end-and-call-macro)
> 1 <- global-set-key: kmacro-end-and-call-macro
> ======================================================================
> 1 -> (global-set-key [f3] kmacro-start-macro-or-insert-counter)
> 1 <- global-set-key: kmacro-start-macro-or-insert-counter
> ======================================================================
> 1 -> (global-set-key [f4] kmacro-end-or-call-macro)
> 1 <- global-set-key: kmacro-end-or-call-macro
> ======================================================================
> 1 -> (global-set-key "\v" kmacro-keymap)
> 1 <- global-set-key: kmacro-keymap
> ======================================================================
> 1 -> (global-set-key "6" 2C-command)
> 1 <- global-set-key: 2C-command
> ======================================================================
> 1 -> (global-set-key [f2] 2C-command)
> 1 <- global-set-key: 2C-command
> 
> which overwrites part of my key bindings, also with ones from two-column.

Those bindings come from loaddefs.el, which should be preloaded, so
the bindings should be executed when dumping Emacs, as part of loading
loadup.el, not when processing your init files.  Can you tell more
about how you captured those traces, exactly?  Also, what are the
details of your build, and in particular did you build
"--with-native-compilation=aot", per chance?

Stefan, any comments?

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Gerd Möllmann <gerd.moellmann@gmail.com>
>> Date: Sat, 16 Sep 2023 08:38:22 +0200
>>
>> current master 1442f4043a761e9bdeeb4e1fbe9822c2987c1502, emacs -Q
>
> The same problem exists on emacs-29 (and I presume in older versions
> of Emacs).
>
>>   (keymap-global-set "S-<mouse-3>" 'ignore)
>>
>> then C-x ( C-g.
>
> (The C-g part is not relevant; the same happens if you finish defining
> the macro.)

True.

>>   (keymap-lookup global-map "S-<mouse-3>")
>>   => kmacro-end-call-mouse
>>
>> Notice that the binding has been overwritten.
>
> This is because of this snippet at top level of kmacro.el:
>
>   (if kmacro-call-mouse-event
>     (global-set-key (vector kmacro-call-mouse-event) #'kmacro-end-call-mouse))
>
> As the doc string of kmacro-call-mouse-event says:
>
>   (defcustom kmacro-call-mouse-event 'S-mouse-3
>     "The mouse event used by kmacro to call a macro.
>   Set to nil if no mouse binding is desired."
>     :type 'symbol)
>
> customize this to nil to avoid overwriting the S-<mouse-3> binding you
> have.

Ah, finally someone I can understand ;-).

>> This is actually not the binding I care about, but it's the only one I
>> can easily reproduce with emacs -Q.  With my init file, and with a
>> (trace-function 'global-set-key) I see
>>
>> ======================================================================
>> 1 -> (global-set-key "(" kmacro-start-macro)
>> 1 <- global-set-key: kmacro-start-macro
>> ======================================================================
>> 1 -> (global-set-key ")" kmacro-end-macro)
>> 1 <- global-set-key: kmacro-end-macro
>> ======================================================================
>> 1 -> (global-set-key "e" kmacro-end-and-call-macro)
>> 1 <- global-set-key: kmacro-end-and-call-macro
>> ======================================================================
>> 1 -> (global-set-key [f3] kmacro-start-macro-or-insert-counter)
>> 1 <- global-set-key: kmacro-start-macro-or-insert-counter
>> ======================================================================
>> 1 -> (global-set-key [f4] kmacro-end-or-call-macro)
>> 1 <- global-set-key: kmacro-end-or-call-macro
>> ======================================================================
>> 1 -> (global-set-key "\v" kmacro-keymap)
>> 1 <- global-set-key: kmacro-keymap
>> ======================================================================
>> 1 -> (global-set-key "6" 2C-command)
>> 1 <- global-set-key: 2C-command
>> ======================================================================
>> 1 -> (global-set-key [f2] 2C-command)
>> 1 <- global-set-key: 2C-command
>>
>> which overwrites part of my key bindings, also with ones from two-column.
>
> Those bindings come from loaddefs.el, which should be preloaded, so
> the bindings should be executed when dumping Emacs, as part of loading
> loadup.el, not when processing your init files.

So I thought, too.

> Can you tell more about how you captured those traces, exactly?

I added a (trace-function 'global-set-key) to my init file, in an
after-init-hook, so that it only traced after my key bindings were done.
At some point, I still don't know how I triggered this, this trace
appeared.

> what are the details of your build, and in particular did you build
> "--with-native-compilation=aot", per chance?

No native compilation, the only config option different fromt he default
is --without-dbus.  Please find at the end what report-emacs-bug
produces.  It's a clean build starting with git clean -xdf.

I have now added this to my init.el:

(defun my-watch-f2 (&rest args)
  (when (eq (car args) global-map)
    (let ((key (cl-second args)))
      (when (and (vectorp key) (eq (aref key 0) 'f2))
	(debug)))))

(add-hook 'after-init-hook
	  (lambda ()
	    (add-function :after (symbol-function 'define-key) #'my-watch-f2)))

which is supposed to drop me into the debugger when my F2 binding gets
redefined.  Alas, as I mentioned, I don't know what exactly I do when
this happens.

I'll report back, when I have something.



In GNU Emacs 30.0.50 (build 1, x86_64-apple-darwin20.6.0, NS
 appkit-2022.70 Version 11.7.10 (Build 20G1427)) of 2023-09-16 built on
 Pro.fritz.box
Repository revision: 302bc23f7cb9fbde7f225650d833e1c09da97338
Repository branch: master
Windowing system distributor 'Apple', version 10.3.2022
System Description:  macOS 11.7.10

Configured using:
 'configure --without-dbus --cache-file
 /var/folders/1d/k_6t25f94sl83szqbf8gpkrh0000gn/T//config.cache.master
 --disable-silent-rules'

Configured features:
ACL GIF GLIB GMP GNUTLS JPEG LCMS2 LIBXML2 MODULES NOTIFY KQUEUE NS
PDUMPER PNG RSVG SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS TREE_SITTER
WEBP XIM ZLIB

Important settings:
  locale-coding-system: utf-8-unix

Major mode: Message

Minor modes in effect:
  global-git-commit-mode: t
  magit-auto-revert-mode: t
  gnus-message-citation-mode: t
  mml-mode: t
  global-company-mode: t
  company-mode: t
  desktop-save-mode: t
  global-auto-revert-mode: t
  recentf-mode: t
  server-mode: t
  pixel-scroll-precision-mode: t
  yas-global-mode: t
  yas-minor-mode: t
  which-key-mode: t
  persistent-scratch-autosave-mode: t
  delete-selection-mode: t
  global-hl-line-mode: t
  global-display-line-numbers-mode: t
  display-line-numbers-mode: t
  ws-butler-global-mode: t
  ws-butler-mode: t
  vertico-posframe-mode: t
  vertico-multiform-mode: t
  vertico-mouse-mode: t
  vertico-grid-mode: t
  vertico-mode: t
  override-global-mode: t
  straight-use-package-mode: t
  straight-package-neutering-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  minibuffer-regexp-mode: t
  line-number-mode: t
  auto-fill-function: yas--auto-fill
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  abbrev-mode: t

Load-path shadows:
/Users/gerd/.config/emacs.d.default/straight/build/external-completion/external-completion hides /Applications/Emacs.app/Contents/Resources/lisp/external-completion
/Users/gerd/.config/emacs.d.default/straight/build/jsonrpc/jsonrpc hides /Applications/Emacs.app/Contents/Resources/lisp/jsonrpc
/Users/gerd/.config/emacs.d.default/straight/build/transient/transient hides /Applications/Emacs.app/Contents/Resources/lisp/transient
/Users/gerd/.config/emacs.d.default/straight/build/xref/xref hides /Applications/Emacs.app/Contents/Resources/lisp/progmodes/xref
/Users/gerd/.config/emacs.d.default/straight/build/project/project hides /Applications/Emacs.app/Contents/Resources/lisp/progmodes/project
/Users/gerd/.config/emacs.d.default/straight/build/eglot/eglot hides /Applications/Emacs.app/Contents/Resources/lisp/progmodes/eglot
/Users/gerd/.config/emacs.d.default/straight/build/flymake/flymake hides /Applications/Emacs.app/Contents/Resources/lisp/progmodes/flymake
/Users/gerd/.config/emacs.d.default/straight/build/soap-client/soap-client hides /Applications/Emacs.app/Contents/Resources/lisp/net/soap-client
/Users/gerd/.config/emacs.d.default/straight/build/soap-client/soap-inspect hides /Applications/Emacs.app/Contents/Resources/lisp/net/soap-inspect
/Users/gerd/.config/emacs.d.default/straight/build/eldoc/eldoc hides /Applications/Emacs.app/Contents/Resources/lisp/emacs-lisp/eldoc

Features:
(shadow emacsbug nndoc gnus-dup mm-archive url-cache debbugs-gnu
debbugs-compat debbugs soap-client url-http url-auth url-gw rng-xsd
rng-dt rng-util xsd-regexp consult-register magit-extras magit-bookmark
magit-submodule magit-blame magit-stash magit-reflog magit-bisect
magit-push magit-pull magit-fetch magit-clone magit-remote magit-commit
magit-sequence magit-notes magit-worktree magit-tag magit-merge
magit-branch magit-reset magit-files magit-refs magit-status magit
package url-handlers magit-repos magit-apply magit-wip magit-log
which-func magit-diff smerge-mode git-commit log-edit add-log magit-core
magit-autorevert magit-margin magit-transient magit-process with-editor
magit-mode magit-git magit-base crm qp sort smiley gnus-cite mail-extr
textsec uni-scripts idna-mapping uni-confusable textsec-check gnus-async
gnus-bcklg gnus-ml disp-table nndraft nnmh utf-7 nnfolder epa-file
network-stream nsm gnus-agent gnus-srvr gnus-score score-mode nnvirtual
gnus-msg gnus-art mm-uu mml2015 mm-view mml-smime smime gnutls dig nntp
gnus-cache gnus-sum shr pixel-fill kinsoku url-file svg dom browse-url
url url-proxy url-privacy url-expand url-methods url-history url-cookie
generate-lisp-file url-domsuf gnus-group gnus-undo gnus-start gnus-dbus
dbus xml gnus-cloud nnimap nnmail mail-source utf7 nnoo parse-time
iso8601 gnus-spec gnus-int gnus-range message mailcap yank-media puny
rfc822 mml mml-sec epa derived epg rfc6068 epg-config mm-decode
mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
gnus-win shortdoc comp comp-cstr rx help-fns radix-tree
exec-path-from-shell consult-eglot cc-mode cc-fonts cc-guess cc-menus
cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs eglot
external-completion jsonrpc flymake-proc flymake thingatpt diff ert ewoc
debug backtrace find-func warnings icons url-util url-parse url-vars
imenu time dired-aux jka-compr company-oddmuse company-keywords
company-etags etags fileloop generator xref company-gtags
company-dabbrev-code company-dabbrev company-files company-clang
company-capf company-cmake company-semantic company-template
company-bbdb company pcase org-bullets-autoloads ediff ediff-merg
ediff-mult ediff-wind ediff-diff ediff-help ediff-init ediff-util
inf-lisp shell pcomplete desktop frameset autorevert filenotify recentf
tree-widget dired dired-loaddefs modus-vivendi-theme modus-themes server
transient pixel-scroll cua-base vc-hg vc-git diff-mode vc-bzr vc-src
vc-sccs vc-svn vc-cvs vc-rcs log-view pcvs-util bug-reference
debbugs-autoloads soap-client-autoloads smtpmail auth-source
password-cache json map sendmail rfc2047 rfc2045 ietf-drums yasnippet
yasnippet-autoloads which-key which-key-autoloads persistent-scratch
persistent-scratch-autoloads delsel hl-line display-line-numbers gnus
nnheader gnus-util time-date mail-utils range mm-util mail-prsvr
wid-edit compile comint ansi-osc ansi-color ring ws-butler
ws-butler-autoloads consult-eglot-autoloads eglot-autoloads
external-completion-autoloads flymake-autoloads project-autoloads
xref-autoloads eldoc-autoloads jsonrpc-autoloads company-autoloads
embark-consult-autoloads embark-autoloads consult-ls-git vc
vc-dispatcher project consult-ls-git-autoloads consult bookmark
text-property-search pp consult-autoloads marginalia-autoloads orderless
orderless-autoloads vertico-posframe posframe cus-load
vertico-posframe-autoloads posframe-autoloads vertico-multiform
vertico-mouse vertico-grid vertico vertico-autoloads magit-section
format-spec eieio eieio-core dash cursor-sensor compat edmacro kmacro
byte-opt magit-autoloads magit-section-autoloads git-commit-autoloads
with-editor-autoloads transient-autoloads dash-autoloads
compat-autoloads info use-package-bind-key bind-key easy-mmode
exec-path-from-shell-autoloads finder-inf use-package-core
straight-autoloads cl-seq cl-extra help-mode straight subr-x cl-macs
cl-loaddefs cl-lib bytecomp byte-compile chemacs gv rmc iso-transl
tooltip cconv eldoc paren electric uniquify ediff-hook vc-hooks
lisp-float-type elisp-mode mwheel term/ns-win ns-win ucs-normalize
mule-util term/common-win touch-screen tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu
timer select scroll-bar mouse jit-lock font-lock syntax font-core
term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads kqueue cocoa ns lcms2
multi-tty make-network-process emacs)

Memory information:
((conses 16 446510 266970) (symbols 48 37802 10)
 (strings 32 136051 12377) (string-bytes 1 4585001) (vectors 16 79623)
 (vector-slots 8 1731485 105152) (floats 8 514 627)
 (intervals 56 2530 840) (buffers 992 36))


>
> Stefan, any comments?

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Gerd Möllmann]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> I'll report back, when I have something.

One addendum, even if it might not be relevant - one never knows.

I'm using use-package (+ straight) in my init.el, and because of that
the bindings are done, behind the scenes, with bind-key.  The
use-package binding f2 looks like (stripped down):

  (use-package emacs
    :bind
    ;; FIXME: The function key bindings sometimes mysteriously
    ;; disappear, and are replaced with the original bindings in
    ;; global-map.
    ("<f2>" . mode-line-other-buffer))

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,  66022@debbugs.gnu.org
> Date: Sun, 17 Sep 2023 08:40:06 +0200
> 
> I'm using use-package (+ straight) in my init.el, and because of that
> the bindings are done, behind the scenes, with bind-key.  The
> use-package binding f2 looks like (stripped down):
> 
>   (use-package emacs
>     :bind
>     ;; FIXME: The function key bindings sometimes mysteriously
>     ;; disappear, and are replaced with the original bindings in
>     ;; global-map.
>     ("<f2>" . mode-line-other-buffer))

What does use-package do when the "package" is "emacs"?  IOW, at what
point in time will the above binding be executed, and what event will
trigger that?

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Gerd Möllmann <gerd.moellmann@gmail.com>
>> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,  66022@debbugs.gnu.org
>> Date: Sun, 17 Sep 2023 08:40:06 +0200
>> 
>> I'm using use-package (+ straight) in my init.el, and because of that
>> the bindings are done, behind the scenes, with bind-key.  The
>> use-package binding f2 looks like (stripped down):
>> 
>>   (use-package emacs
>>     :bind
>>     ;; FIXME: The function key bindings sometimes mysteriously
>>     ;; disappear, and are replaced with the original bindings in
>>     ;; global-map.
>>     ("<f2>" . mode-line-other-buffer))
>
> What does use-package do when the "package" is "emacs"?  IOW, at what
> point in time will the above binding be executed, and what event will
> trigger that?

Nothing.  It's sort of an idiom I found on the net for general
customizations and so on that don't belong to a particular package.
Some people apparently name it "general".

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Cc: monnier@iro.umontreal.ca,  66022@debbugs.gnu.org
> Date: Sun, 17 Sep 2023 08:52:07 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> >> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,  66022@debbugs.gnu.org
> >> Date: Sun, 17 Sep 2023 08:40:06 +0200
> >> 
> >> I'm using use-package (+ straight) in my init.el, and because of that
> >> the bindings are done, behind the scenes, with bind-key.  The
> >> use-package binding f2 looks like (stripped down):
> >> 
> >>   (use-package emacs
> >>     :bind
> >>     ;; FIXME: The function key bindings sometimes mysteriously
> >>     ;; disappear, and are replaced with the original bindings in
> >>     ;; global-map.
> >>     ("<f2>" . mode-line-other-buffer))
> >
> > What does use-package do when the "package" is "emacs"?  IOW, at what
> > point in time will the above binding be executed, and what event will
> > trigger that?
> 
> Nothing.  It's sort of an idiom I found on the net for general
> customizations and so on that don't belong to a particular package.
> Some people apparently name it "general".

So the above binding of F2 is never executed in your case?  Or else,
how should I interpret your "Nothing" response to my question about
the point in time when this binding is executed?

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

>> > What does use-package do when the "package" is "emacs"?  IOW, at what
>> > point in time will the above binding be executed, and what event will
>> > trigger that?
>> 
>> Nothing.  It's sort of an idiom I found on the net for general
>> customizations and so on that don't belong to a particular package.
>> Some people apparently name it "general".
>
> So the above binding of F2 is never executed in your case?  Or else,
> how should I interpret your "Nothing" response to my question about
> the point in time when this binding is executed?

No, sorry for not being clear.  I meant use-package doesn't load a
package, or something like that.  It executes normally, and in
particular, it uses bind-key for the :bind clause, so that F2 gets
bound.

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Cc: monnier@iro.umontreal.ca,  66022@debbugs.gnu.org
> Date: Sun, 17 Sep 2023 10:58:11 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> > What does use-package do when the "package" is "emacs"?  IOW, at what
> >> > point in time will the above binding be executed, and what event will
> >> > trigger that?
> >> 
> >> Nothing.  It's sort of an idiom I found on the net for general
> >> customizations and so on that don't belong to a particular package.
> >> Some people apparently name it "general".
> >
> > So the above binding of F2 is never executed in your case?  Or else,
> > how should I interpret your "Nothing" response to my question about
> > the point in time when this binding is executed?
> 
> No, sorry for not being clear.  I meant use-package doesn't load a
> package, or something like that.  It executes normally, and in
> particular, it uses bind-key for the :bind clause, so that F2 gets
> bound.

Does it do that immediately as that particular use-package form is
read and interpreted, or does it do it later (e.g., after the init
file was loaded or something)?

I'm trying to figure out when could the autoloaded bindings become in
effect so as to countermand your use-package bindings (if this is what
happens; does it?).

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Gerd Möllmann <gerd.moellmann@gmail.com>
>> Cc: monnier@iro.umontreal.ca,  66022@debbugs.gnu.org
>> Date: Sun, 17 Sep 2023 10:58:11 +0200
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> > What does use-package do when the "package" is "emacs"?  IOW, at what
>> >> > point in time will the above binding be executed, and what event will
>> >> > trigger that?
>> >> 
>> >> Nothing.  It's sort of an idiom I found on the net for general
>> >> customizations and so on that don't belong to a particular package.
>> >> Some people apparently name it "general".
>> >
>> > So the above binding of F2 is never executed in your case?  Or else,
>> > how should I interpret your "Nothing" response to my question about
>> > the point in time when this binding is executed?
>> 
>> No, sorry for not being clear.  I meant use-package doesn't load a
>> package, or something like that.  It executes normally, and in
>> particular, it uses bind-key for the :bind clause, so that F2 gets
>> bound.
>
> Does it do that immediately as that particular use-package form is
> read and interpreted, or does it do it later (e.g., after the init
> file was loaded or something)?
>
> I'm trying to figure out when could the autoloaded bindings become in
> effect so as to countermand your use-package bindings (if this is what
> happens; does it?).

Hm, I must confess I'm not a great use-package expert, but maybe we can
figure this out.  I used pp-macro-expand-last-sexp on the use-package in
my .init.el.  The result is at the end.

I'd say it looks like the bind-key is done like so

  (condition-case-unless-debug err
      (progn
        ...
	(unless (fboundp 'mode-line-other-buffer)
	  (autoload #'mode-line-other-buffer "emacs" nil t))
        ...
        (bind-key ...)

This is the full macro expansion:

(progn
  (defvar use-package--warning47
    #'(lambda (keyword err)
	(let
	    ((msg
	      (format "%s/%s: %s" 'emacs keyword
		      (error-message-string err))))
	  (display-warning 'use-package msg :error))))
  (condition-case-unless-debug err
      (progn
	(let ((custom--inhibit-theme-enable nil))
	  (unless (memq 'use-package custom-known-themes)
	    (deftheme use-package) (enable-theme 'use-package)
	    (setq custom-enabled-themes
		  (remq 'use-package custom-enabled-themes)))
	  (custom-theme-set-variables 'use-package
				      '(cursor-type 'bar nil nil
						    "Customized with use-package emacs")
				      '(scroll-bar-mode nil nil nil
							"Customized with use-package emacs")
				      '(custom-enabled-themes
					'(modus-vivendi) nil nil
					"Customized with use-package emacs")
				      '(default-frame-alist
					'((height . 82) (width . 120))
					nil nil
					"Customized with use-package emacs")
				      '(ns-alternate-modifier 'meta
							      nil nil
							      "Customized with use-package emacs")
				      '(ns-left-command-modifier
					'super nil nil
					"Customized with use-package emacs")
				      '(ns-right-alternate-modifier
					'none nil nil
					"Customized with use-package emacs")
				      '(ns-right-command-modifier
					'meta nil nil
					"Customized with use-package emacs")
				      '(duplicate-line-final-position
					1 nil nil
					"Customized with use-package emacs")
				      '(enable-local-eval t nil nil
							  "Customized with use-package emacs")
				      '(enable-local-variables :all
							       nil nil
							       "Customized with use-package emacs")
				      '(help-window-select t nil nil
							   "Customized with use-package emacs")
				      '(server-mode t nil nil
						    "Customized with use-package emacs")
				      '(send-mail-function
					'smtpmail-send-it nil nil
					"Customized with use-package emacs")
				      '(vc-follow-symlinks t nil nil
							   "Customized with use-package emacs")
				      '(disabled-command-function nil
								  nil
								  nil
								  "Customized with use-package emacs")
				      '(ring-bell-function 'ignore nil
							   nil
							   "Customized with use-package emacs")
				      '(initial-scratch-message nil
								nil
								nil
								"Customized with use-package emacs")))
	(unless (fboundp 'undo) (autoload #'undo "emacs" nil t))
	(unless (fboundp 'undo-redo)
	  (autoload #'undo-redo "emacs" nil t))
	(unless (fboundp 'ibuffer) (autoload #'ibuffer "emacs" nil t))
	(unless (fboundp 'zap-up-to-char)
	  (autoload #'zap-up-to-char "emacs" nil t))
	(unless (fboundp 'mode-line-other-buffer)
	  (autoload #'mode-line-other-buffer "emacs" nil t))
	(unless (fboundp 'duplicate-dwim)
	  (autoload #'duplicate-dwim "emacs" nil t))
	(unless (fboundp 'switch-to-buffer)
	  (autoload #'switch-to-buffer "emacs" nil t))
	(unless (fboundp 'beginning-of-buffer)
	  (autoload #'beginning-of-buffer "emacs" nil t))
	(unless (fboundp 'end-of-buffer)
	  (autoload #'end-of-buffer "emacs" nil t))
	(unless (fboundp 'previous-buffer)
	  (autoload #'previous-buffer "emacs" nil t))
	(unless (fboundp 'next-buffer)
	  (autoload #'next-buffer "emacs" nil t))
	(unless (fboundp 'execute-extended-command)
	  (autoload #'execute-extended-command "emacs" nil t))
	(unless (fboundp 'delete-other-windows)
	  (autoload #'delete-other-windows "emacs" nil t))
	(unless (fboundp 'global-text-scale-adjust)
	  (autoload #'global-text-scale-adjust "emacs" nil t))
	(eval-after-load 'emacs
	  '(condition-case-unless-debug err
	       (progn
		 (defun my-startup-time nil
		   (message "Start time %s." (emacs-init-time)))
		 (add-hook 'emacs-startup-hook #'my-startup-time) t)
	     (error (funcall use-package--warning47 :config err))))
	(bind-keys :package emacs ("C-z" . undo) ("C-S-z" . undo-redo)
		   ("C-x C-b" . ibuffer) ("C-x 6") ("C-x C-k")
		   ("M-z" . zap-up-to-char)
		   ("<f2>" . mode-line-other-buffer)
		   ("C-c o" . mode-line-other-buffer)
		   ("C-c C-o" . mode-line-other-buffer)
		   ("<f4>" lambda nil (interactive)
		    (kill-buffer (current-buffer)))
		   ("C-c k" lambda nil (interactive)
		    (kill-buffer (current-buffer)))
		   ("C-c C-k" lambda nil (interactive)
		    (kill-buffer (current-buffer)))
		   ("C-<up>" lambda nil (interactive) (scroll-up 1))
		   ("C-<down>" lambda nil (interactive)
		    (scroll-down 1))
		   ("C-<tab>" . other-window)
		   ("C-c d" . duplicate-dwim) (" " . switch-to-buffer)
		   ("s-<up>" . beginning-of-buffer)
		   ("s-<down>" . end-of-buffer)
		   ("s-<left>" . previous-buffer)
		   ("s-<right>" . next-buffer)
		   ("s-x" . execute-extended-command)
		   ("s-o" . delete-other-windows)
		   ("s-+" . global-text-scale-adjust)
		   ("s--" . global-text-scale-adjust) ("s-h") ("s-m")
		   ("s-p") ("s-q")))
    (error (funcall use-package--warning47 :catch err))))

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

>
> I'm trying to figure out when could the autoloaded bindings become in
> effect so as to countermand your use-package bindings (if this is what
> happens; does it?).

Sorry, forgot to adress that question.

Yes, that's what is happening.  Everything works normally for a long
time, I do whatever it is in Emacs, and at some point I notice f2
isn't working as before.

ALas, even when the trace-function kicked in, I wasn't able to tell what
exactly it was that triggered that.

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Cc: monnier@iro.umontreal.ca,  66022@debbugs.gnu.org
> Date: Sun, 17 Sep 2023 12:02:16 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >
> > I'm trying to figure out when could the autoloaded bindings become in
> > effect so as to countermand your use-package bindings (if this is what
> > happens; does it?).
> 
> Sorry, forgot to adress that question.
> 
> Yes, that's what is happening.  Everything works normally for a long
> time, I do whatever it is in Emacs, and at some point I notice f2
> isn't working as before.
> 
> ALas, even when the trace-function kicked in, I wasn't able to tell what
> exactly it was that triggered that.

Could it be that you (or something else) reloaded loaddefs?

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

>> ALas, even when the trace-function kicked in, I wasn't able to tell what
>> exactly it was that triggered that.
>
> Could it be that you (or something else) reloaded loaddefs?

It wasn't me, for sure, but that's what I also suspect is happening
somehow.  An indicator for that is that the kmacro and two-column
bindings appeared together.  These are the only 2 packages putting
global-set-keys into loaddefs.

But how would that happen if loaddefs is preloaded?  I couldn't find
anything suspicous with git grep.

I'm building with ASAN now, maybe that finds something.

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Cc: monnier@iro.umontreal.ca,  66022@debbugs.gnu.org
> Date: Sun, 17 Sep 2023 13:07:47 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Could it be that you (or something else) reloaded loaddefs?
> 
> It wasn't me, for sure, but that's what I also suspect is happening
> somehow.  An indicator for that is that the kmacro and two-column
> bindings appeared together.  These are the only 2 packages putting
> global-set-keys into loaddefs.

Maybe try using eval-after-load to catch the culprit?

load-history should also tell you if loaddefs was loaded, I think.

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Gerd Möllmann]

On 23-09-17 13:23 , Eli Zaretskii wrote:
>> From: Gerd Möllmann <gerd.moellmann@gmail.com>
>> Cc: monnier@iro.umontreal.ca,  66022@debbugs.gnu.org
>> Date: Sun, 17 Sep 2023 13:07:47 +0200
>>
>> Eli Zaretskii <eliz@gnu.org> writes:
>>
>>> Could it be that you (or something else) reloaded loaddefs?
>>
>> It wasn't me, for sure, but that's what I also suspect is happening
>> somehow.  An indicator for that is that the kmacro and two-column
>> bindings appeared together.  These are the only 2 packages putting
>> global-set-keys into loaddefs.
> 
> Maybe try using eval-after-load to catch the culprit?

That's a good idea!  I'll put that in my init.el.

The ASAN build on my old Macbook is so slow, it's no fun.

> load-history should also tell you if loaddefs was loaded, I think.

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> But how would that happen if loaddefs is preloaded?

One case this can happen is if you're hacking on the Emacs you're
currently running: if you look at the doc of a function, Emacs
will fetch that doc from the `.elc` file, but if that file was
regenerated and the doc has moved Emacs usually detects the problem and
reloads the (new) `.elc` file so as to get the new position.


        Stefan

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Gerd Möllmann]

On 23-09-17 19:13 , Stefan Monnier wrote:
>> But how would that happen if loaddefs is preloaded?
> 
> One case this can happen is if you're hacking on the Emacs you're 
> currently running: if you look at the doc of a function, Emacs will 
> fetch that doc from the `.elc` file, but if that file was
> regenerated and the doc has moved Emacs usually detects the problem
> and reloads the (new) `.elc` file so as to get the new position.

Thanks, that's an idea.

I don't remember doing that, but I'll try to reproduce this somehow.  I
tried it just now with a simple recipe but that didn't trigger the
checks in my init.el.  Have to read the code that does that reloading, I
guess.

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> I don't remember doing that, but I'll try to reproduce this somehow.  I
> tried it just now with a simple recipe but that didn't trigger the
> checks in my init.el.  Have to read the code that does that reloading, I
> guess.

grep 'Fload (' src/*.c

should get you started.


        Stefan :-)

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Gerd Möllmann]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> I don't remember doing that, but I'll try to reproduce this somehow.  I
>> tried it just now with a simple recipe but that didn't trigger the
>> checks in my init.el.  Have to read the code that does that reloading, I
>> guess.
>
> grep 'Fload (' src/*.c
>
> should get you started.

Caught the sucker :-).  I think your intuition was right.

This is Emacs.app, with my init.el.  I simply did a C-c, which-key
kicked in, calls documentation, and here we are...

Debugger entered: nil
  (progn (debug))
  (if (and (vectorp key) (eq (aref key 0) 'f2)) (progn (debug)))
  (let ((key (car (cdr args)))) (if (and (vectorp key) (eq (aref key 0) 'f2)) (progn (debug))))
  (progn (let ((key (car (cdr args)))) (if (and (vectorp key) (eq (aref key 0) 'f2)) (progn (debug)))))
  (if (eq (car args) global-map) (progn (let ((key (car (cdr args)))) (if (and (vectorp key) (eq (aref key 0) 'f2)) (progn (debug))))))
  my-watch-f2((keymap #^[nil nil keymap 
#^^[3 0 set-mark-command move-beginning-of-line backward-char mode-specific-command-prefix delete-char move-end-of-line forward-char keyboard-quit help-command indent-for-tab-command electric-newline-and-maybe-indent kill-line recenter-top-bottom newline next-line open-line previous-line quoted-insert isearch-backward isearch-forward transpose-chars universal-argument scroll-up-command kill-region Control-X-prefix yank undo ESC-prefix toggle-input-method abort-recursive-edit nil undo self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command ...] #^^[1 0 #^^[2 0 
#^^[3 0 set-mark-command move-beginning-of-line backward-char mode-specific-command-prefix delete-char move-end-of-line forward-char keyboard-quit help-command indent-for-tab-command electric-newline-and-maybe-indent kill-line recenter-top-bottom newline next-line open-line previous-line quoted-insert isearch-backward isearch-forward transpose-chars universal-argument scroll-up-command kill-region Control-X-prefix yank undo ESC-prefix toggle-input-method abort-recursive-edit nil undo self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command ...] 
#^^[3 128 self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command switch-to-buffer self-insert-command self-insert-command self-insert-command self-insert-command ...] self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command] self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command] self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command ...] (C-drag-mouse-1 . ignore) (s-down . end-of-buffer) (s-up . beginning-of-buffer) (C-tab . other-window) (33554458 . undo-redo) (67108899 . embark-act) (S-mouse-3 . kmacro-end-call-mouse) (C-mouse-5 . mouse-wheel-text-scale) (C-mouse-4 . mouse-wheel-text-scale) (C-wheel-down . mouse-wheel-text-scale) (C-wheel-up . mouse-wheel-text-scale) (C-M-mouse-5 . mouse-wheel-global-text-scale) (C-M-mouse-4 . mouse-wheel-global-text-scale) (C-M-wheel-down . mouse-wheel-global-text-scale) (C-M-wheel-up . mouse-wheel-global-text-scale) (M-mouse-7 . mwheel-scroll) (M-mouse-6 . mwheel-scroll) (M-mouse-5 . mwheel-scroll) (M-mouse-4 . mwheel-scroll) (M-wheel-right . mwheel-scroll) (M-wheel-left . mwheel-scroll) (M-wheel-down . mwheel-scroll) (M-wheel-up . mwheel-scroll) (S-mouse-7 . mwheel-scroll) (S-mouse-6 . mwheel-scroll) (S-mouse-5 . mwheel-scroll) (S-mouse-4 . mwheel-scroll) (S-wheel-right . mwheel-scroll) (S-wheel-left . mwheel-scroll) (S-wheel-down . mwheel-scroll) (S-wheel-up . mwheel-scroll) (mouse-7 . mwheel-scroll) (mouse-6 . mwheel-scroll) (mouse-5 . mwheel-scroll) (mouse-4 . mwheel-scroll) (wheel-right . mwheel-scroll) ...) [f2] 2C-command)
  apply(my-watch-f2 ((keymap #^[nil nil keymap 
#^^[3 0 set-mark-command move-beginning-of-line backward-char mode-specific-command-prefix delete-char move-end-of-line forward-char keyboard-quit help-command indent-for-tab-command electric-newline-and-maybe-indent kill-line recenter-top-bottom newline next-line open-line previous-line quoted-insert isearch-backward isearch-forward transpose-chars universal-argument scroll-up-command kill-region Control-X-prefix yank undo ESC-prefix toggle-input-method abort-recursive-edit nil undo self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command ...] #^^[1 0 #^^[2 0 
#^^[3 0 set-mark-command move-beginning-of-line backward-char mode-specific-command-prefix delete-char move-end-of-line forward-char keyboard-quit help-command indent-for-tab-command electric-newline-and-maybe-indent kill-line recenter-top-bottom newline next-line open-line previous-line quoted-insert isearch-backward isearch-forward transpose-chars universal-argument scroll-up-command kill-region Control-X-prefix yank undo ESC-prefix toggle-input-method abort-recursive-edit nil undo self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command ...] 
#^^[3 128 self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command switch-to-buffer self-insert-command self-insert-command self-insert-command self-insert-command ...] self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command] self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command] self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command ...] (C-drag-mouse-1 . ignore) (s-down . end-of-buffer) (s-up . beginning-of-buffer) (C-tab . other-window) (33554458 . undo-redo) (67108899 . embark-act) (S-mouse-3 . kmacro-end-call-mouse) (C-mouse-5 . mouse-wheel-text-scale) (C-mouse-4 . mouse-wheel-text-scale) (C-wheel-down . mouse-wheel-text-scale) (C-wheel-up . mouse-wheel-text-scale) (C-M-mouse-5 . mouse-wheel-global-text-scale) (C-M-mouse-4 . mouse-wheel-global-text-scale) (C-M-wheel-down . mouse-wheel-global-text-scale) (C-M-wheel-up . mouse-wheel-global-text-scale) (M-mouse-7 . mwheel-scroll) (M-mouse-6 . mwheel-scroll) (M-mouse-5 . mwheel-scroll) (M-mouse-4 . mwheel-scroll) (M-wheel-right . mwheel-scroll) (M-wheel-left . mwheel-scroll) (M-wheel-down . mwheel-scroll) (M-wheel-up . mwheel-scroll) (S-mouse-7 . mwheel-scroll) (S-mouse-6 . mwheel-scroll) (S-mouse-5 . mwheel-scroll) (S-mouse-4 . mwheel-scroll) (S-wheel-right . mwheel-scroll) (S-wheel-left . mwheel-scroll) (S-wheel-down . mwheel-scroll) (S-wheel-up . mwheel-scroll) (mouse-7 . mwheel-scroll) (mouse-6 . mwheel-scroll) (mouse-5 . mwheel-scroll) (mouse-4 . mwheel-scroll) (wheel-right . mwheel-scroll) ...) [f2] 2C-command))
  define-key((keymap #^[nil nil keymap 
#^^[3 0 set-mark-command move-beginning-of-line backward-char mode-specific-command-prefix delete-char move-end-of-line forward-char keyboard-quit help-command indent-for-tab-command electric-newline-and-maybe-indent kill-line recenter-top-bottom newline next-line open-line previous-line quoted-insert isearch-backward isearch-forward transpose-chars universal-argument scroll-up-command kill-region Control-X-prefix yank undo ESC-prefix toggle-input-method abort-recursive-edit nil undo self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command ...] #^^[1 0 #^^[2 0 
#^^[3 0 set-mark-command move-beginning-of-line backward-char mode-specific-command-prefix delete-char move-end-of-line forward-char keyboard-quit help-command indent-for-tab-command electric-newline-and-maybe-indent kill-line recenter-top-bottom newline next-line open-line previous-line quoted-insert isearch-backward isearch-forward transpose-chars universal-argument scroll-up-command kill-region Control-X-prefix yank undo ESC-prefix toggle-input-method abort-recursive-edit nil undo self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command ...] 
#^^[3 128 self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command switch-to-buffer self-insert-command self-insert-command self-insert-command self-insert-command ...] self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command] self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command] self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command self-insert-command ...] (C-drag-mouse-1 . ignore) (s-down . end-of-buffer) (s-up . beginning-of-buffer) (C-tab . other-window) (33554458 . undo-redo) (67108899 . embark-act) (S-mouse-3 . kmacro-end-call-mouse) (C-mouse-5 . mouse-wheel-text-scale) (C-mouse-4 . mouse-wheel-text-scale) (C-wheel-down . mouse-wheel-text-scale) (C-wheel-up . mouse-wheel-text-scale) (C-M-mouse-5 . mouse-wheel-global-text-scale) (C-M-mouse-4 . mouse-wheel-global-text-scale) (C-M-wheel-down . mouse-wheel-global-text-scale) (C-M-wheel-up . mouse-wheel-global-text-scale) (M-mouse-7 . mwheel-scroll) (M-mouse-6 . mwheel-scroll) (M-mouse-5 . mwheel-scroll) (M-mouse-4 . mwheel-scroll) (M-wheel-right . mwheel-scroll) (M-wheel-left . mwheel-scroll) (M-wheel-down . mwheel-scroll) (M-wheel-up . mwheel-scroll) (S-mouse-7 . mwheel-scroll) (S-mouse-6 . mwheel-scroll) (S-mouse-5 . mwheel-scroll) (S-mouse-4 . mwheel-scroll) (S-wheel-right . mwheel-scroll) (S-wheel-left . mwheel-scroll) (S-wheel-down . mwheel-scroll) (S-wheel-up . mwheel-scroll) (mouse-7 . mwheel-scroll) (mouse-6 . mwheel-scroll) (mouse-5 . mwheel-scroll) (mouse-4 . mwheel-scroll) (wheel-right . mwheel-scroll) ...) [f2] 2C-command)
  global-set-key([f2] 2C-command)
  byte-code("\300\301\302\"\210\303\304\305\306\307\310%\210\311\312\304\"\210\311\313\304\"\207" [register-definition-prefixes "tv-util" ("tai-viet-") autoload 2C-command "two-column" nil t keymap global-set-key "\0306" [f2]] 6)
  documentation(duplicate-dwim)
  which-key--propertize-description("duplicate-dwim" nil nil nil "duplicate-dwim")
  which-key--format-and-replace((("C-c #" . "embark-act") ("C-c &" . "prefix") ("C-c d" . "duplicate-dwim") ("C-c g" . "my-grab-bag") ("C-c k" . "lambda") ("C-c m" . "magit-file-dispatch") ("C-c o" . "mode-line-other-buffer") ("C-c v" . "prefix") ("C-c C-b" . "elisp-byte-compile-buffer") ("C-c C-e" . "elisp-eval-region-or-buffer") ("C-c C-f" . "elisp-byte-compile-file") ("C-c C-k" . "lambda") ("C-c C-o" . "mode-line-other-buffer") ("C-c <tab>" . "company-complete")) nil)
  which-key--get-bindings([3] nil nil)
  which-key--create-buffer-and-show([3])
  which-key--update()
  apply(which-key--update nil)
  timer-event-handler([t 0 1 0 t which-key--update nil idle 0 nil])

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Gerd Möllmann]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
>>> I don't remember doing that, but I'll try to reproduce this somehow.  I
>>> tried it just now with a simple recipe but that didn't trigger the
>>> checks in my init.el.  Have to read the code that does that reloading, I
>>> guess.
>>
>> grep 'Fload (' src/*.c
>>
>> should get you started.
>
> Caught the sucker :-).  I think your intuition was right.

Here's the backtrace from the second watchdog, the one watching loaddefs
being loaded

Debugger entered: nil
  (if my-debug-on-loaddefs (debug) (setq my-debug-on-loaddefs t))
  my-debug-on-loaddefs()
  (lambda nil (my-debug-on-loaddefs))()
  eval-after-load-helper("/Applications/Emacs.app/Contents/Resources/lisp/lo...")
  run-hook-with-args(eval-after-load-helper "/Applications/Emacs.app/Contents/Resources/lisp/lo...")
  do-after-load-evaluation("/Applications/Emacs.app/Contents/Resources/lisp/lo...")
  documentation(duplicate-dwim)
  which-key--propertize-description("duplicate-dwim" nil nil nil "duplicate-dwim")
  which-key--format-and-replace((("C-c #" . "embark-act") ("C-c &" . "prefix") ("C-c d" . "duplicate-dwim") ("C-c g" . "my-grab-bag") ("C-c k" . "lambda") ("C-c m" . "magit-file-dispatch") ("C-c o" . "mode-line-other-buffer") ("C-c v" . "prefix") ("C-c C-b" . "elisp-byte-compile-buffer") ("C-c C-e" . "elisp-eval-region-or-buffer") ("C-c C-f" . "elisp-byte-compile-file") ("C-c C-k" . "lambda") ("C-c C-o" . "mode-line-other-buffer") ("C-c <tab>" . "company-complete")) nil)
  which-key--get-bindings([3] nil nil)
  which-key--create-buffer-and-show([3])
  which-key--update()
  apply(which-key--update nil)
  timer-event-handler([t 0 1 0 t which-key--update nil idle 0 nil])

and here is the code I have in my init.el

(defun my-watch-f2 (&rest args)
  (when (eq (car args) global-map)
    (let ((key (cl-second args)))
      (when (and (vectorp key) (eq (aref key 0) 'f2))
	(debug)))))

(add-hook 'after-init-hook
	  (lambda ()
	    (add-function :after (symbol-function 'define-key) #'my-watch-f2)))

(defvar my-debug-on-loaddefs nil)

(defun my-debug-on-loaddefs ()
  (if my-debug-on-loaddefs
      (debug)
    (setq my-debug-on-loaddefs t)))

(with-eval-after-load 'loaddefs
  (my-debug-on-loaddefs))

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Cc: Eli Zaretskii <eliz@gnu.org>,  66022@debbugs.gnu.org
> Date: Mon, 18 Sep 2023 16:42:17 +0200
> 
> Here's the backtrace from the second watchdog, the one watching loaddefs
> being loaded
> 
> Debugger entered: nil
>   (if my-debug-on-loaddefs (debug) (setq my-debug-on-loaddefs t))
>   my-debug-on-loaddefs()
>   (lambda nil (my-debug-on-loaddefs))()
>   eval-after-load-helper("/Applications/Emacs.app/Contents/Resources/lisp/lo...")
>   run-hook-with-args(eval-after-load-helper "/Applications/Emacs.app/Contents/Resources/lisp/lo...")
>   do-after-load-evaluation("/Applications/Emacs.app/Contents/Resources/lisp/lo...")
>   documentation(duplicate-dwim)

Any idea why a call to 'documentation' loads loaddefs?  Did you indeed
"hack the Emacs you were running"?

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Gerd Möllmann <gerd.moellmann@gmail.com>
>> Cc: Eli Zaretskii <eliz@gnu.org>,  66022@debbugs.gnu.org
>> Date: Mon, 18 Sep 2023 16:42:17 +0200
>> 
>> Here's the backtrace from the second watchdog, the one watching loaddefs
>> being loaded
>> 
>> Debugger entered: nil
>>   (if my-debug-on-loaddefs (debug) (setq my-debug-on-loaddefs t))
>>   my-debug-on-loaddefs()
>>   (lambda nil (my-debug-on-loaddefs))()
>>   eval-after-load-helper("/Applications/Emacs.app/Contents/Resources/lisp/lo...")
>>   run-hook-with-args(eval-after-load-helper "/Applications/Emacs.app/Contents/Resources/lisp/lo...")
>>   do-after-load-evaluation("/Applications/Emacs.app/Contents/Resources/lisp/lo...")
>>   documentation(duplicate-dwim)
>
> Any idea why a call to 'documentation' loads loaddefs?  Did you indeed
> "hack the Emacs you were running"?

No idea, and I didn't hack it at all.  This was a minute after I started
a newly-build Emacs, wondered why a C-c C-k binding didn't work.  I
then pressed C-c alone to see what it is bound to, with which-key.

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Cc: monnier@iro.umontreal.ca,  66022@debbugs.gnu.org
> Date: Mon, 18 Sep 2023 16:56:55 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Any idea why a call to 'documentation' loads loaddefs?  Did you indeed
> > "hack the Emacs you were running"?
> 
> No idea, and I didn't hack it at all.  This was a minute after I started
> a newly-build Emacs, wondered why a C-c C-k binding didn't work.  I
> then pressed C-c alone to see what it is bound to, with which-key.

So I think this now becomes the main question.  If this is not a bug,
i.e. if loaddefs can legitimately be loaded mid-session, we cannot
autoload key bindings, at least not naïvely as we do now.

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Gerd Möllmann]

On 23-09-18 17:09 , Eli Zaretskii wrote:
>> From: Gerd Möllmann <gerd.moellmann@gmail.com> Cc:
>> monnier@iro.umontreal.ca,  66022@debbugs.gnu.org Date: Mon, 18 Sep
>> 2023 16:56:55 +0200
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>>> Any idea why a call to 'documentation' loads loaddefs?  Did you
>>> indeed "hack the Emacs you were running"?
>> 
>> No idea, and I didn't hack it at all.  This was a minute after I
>> started a newly-build Emacs, wondered why a C-c C-k binding didn't
>> work.  I then pressed C-c alone to see what it is bound to, with
>> which-key.
> 
> So I think this now becomes the main question.  If this is not a
> bug, i.e. if loaddefs can legitimately be loaded mid-session, we
> cannot autoload key bindings, at least not naïvely as we do now.

Another observation: when I start emacs from its source directory
master/src, and not as bundle, i.e. by® clicking on Emacs.app in the
dock, this doesn't happen.

One difference in the two scenarios is that Emacs.app contains gzip'd el
files.

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Gerd Möllmann]

On 23-09-18 17:18 , Gerd Möllmann wrote:
> On 23-09-18 17:09 , Eli Zaretskii wrote:
>>> From: Gerd Möllmann <gerd.moellmann@gmail.com> Cc:
>>> monnier@iro.umontreal.ca,  66022@debbugs.gnu.org Date: Mon, 18 Sep
>>> 2023 16:56:55 +0200
>>>
>>> Eli Zaretskii <eliz@gnu.org> writes:
>>>
>>>> Any idea why a call to 'documentation' loads loaddefs?  Did you
>>>> indeed "hack the Emacs you were running"?
>>>
>>> No idea, and I didn't hack it at all.  This was a minute after I
>>> started a newly-build Emacs, wondered why a C-c C-k binding didn't
>>> work.  I then pressed C-c alone to see what it is bound to, with
>>> which-key.
>>
>> So I think this now becomes the main question.  If this is not a
>> bug, i.e. if loaddefs can legitimately be loaded mid-session, we
>> cannot autoload key bindings, at least not naïvely as we do now.
> 
> Another observation: when I start emacs from its source directory
> master/src, and not as bundle, i.e. by® clicking on Emacs.app in the
> dock, this doesn't happen.
> 
> One difference in the two scenarios is that Emacs.app contains gzip'd el
> files.

~/emacs/master/ > ls -l 
/Applications/Emacs.app/Contents/Resources/lisp/loaddefs.el*
-rw-r--r--  1 gerd  admin   377687 Sep 17 16:36 
/Applications/Emacs.app/Contents/Resources/lisp/loaddefs.el.gz
-rw-r--r--  1 gerd  admin  1467107 Sep 17 16:38 
/Applications/Emacs.app/Contents/Resource

Note the timestamps.  Is that good or bad?

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> Any idea why a call to 'documentation' loads loaddefs?  Did you indeed
>> "hack the Emacs you were running"?
> No idea, and I didn't hack it at all.  This was a minute after I started
> a newly-build Emacs, wondered why a C-c C-k binding didn't work.

Hmm... sounds like we have a real bug, then!


        Stefan

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> ~/emacs/master/ > ls -l
> /Applications/Emacs.app/Contents/Resources/lisp/loaddefs.el*
> -rw-r--r--  1 gerd  admin   377687 Sep 17 16:36
>  /Applications/Emacs.app/Contents/Resources/lisp/loaddefs.el.gz
> -rw-r--r--  1 gerd  admin  1467107 Sep 17 16:38
>  /Applications/Emacs.app/Contents/Resource

Hmm... shouldn't there be a `loaddefs.elc` file in there?


        Stefan

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Gerd Möllmann]

Stefan Monnier via "Bug reports for GNU Emacs, the Swiss army knife of
text editors" <bug-gnu-emacs@gnu.org> writes:

>> ~/emacs/master/ > ls -l
>> /Applications/Emacs.app/Contents/Resources/lisp/loaddefs.el*
>> -rw-r--r--  1 gerd  admin   377687 Sep 17 16:36
>>  /Applications/Emacs.app/Contents/Resources/lisp/loaddefs.el.gz
>> -rw-r--r--  1 gerd  admin  1467107 Sep 17 16:38
>>  /Applications/Emacs.app/Contents/Resource
>
> Hmm... shouldn't there be a `loaddefs.elc` file in there?

Sorry, missed copying a line from the terminal

/ >  ls -l /Applications/Emacs.app/Contents/Resources/lisp/loaddefs.el* 
-rw-r--r--  1 gerd  admin   377687 Sep 17 16:36 /Applications/Emacs.app/Contents/Resources/lisp/loaddefs.el.gz
-rw-r--r--  1 gerd  admin  1467107 Sep 17 16:38 /Applications/Emacs.app/Contents/Resources/lisp/loaddefs.elc
 gerd@Pro 2023-09-18 18:32 
~/ > 

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> Sorry, missed copying a line from the terminal
>
> / >  ls -l /Applications/Emacs.app/Contents/Resources/lisp/loaddefs.el* 
> -rw-r--r--  1 gerd  admin   377687 Sep 17 16:36
> /Applications/Emacs.app/Contents/Resources/lisp/loaddefs.el.gz
> -rw-r--r--  1 gerd  admin  1467107 Sep 17 16:38
> /Applications/Emacs.app/Contents/Resources/lisp/loaddefs.elc
>  gerd@Pro 2023-09-18 18:32 
> ~/ > 

I believe the `lread.c` code which calls `Fload` should be looking at
the `loaddefs.elc` file here, so removing the `.el.gz` should make
no difference.

Now why does it feel the need to reload this file?  The way the code
works is that it receives a (FILE . POS) pair where FILE should
presumably point to the `loaddefs.elc`.

When it opens that `.elc` file, POS should point to the beginning of the
desired docstring with nearby info indicating the length of
that docstring.

If the file was changed in the mean time, POS may end up
pointing elsewhere.  `lread.c` performs a sanity check to make sure POS
points to something that does indeed look like a docstring and
apparently in your case that sanity check fails.

My crystal ball suggests maybe you're witnessing a bug in the build
process where we end up *re*compiling `loaddefs.elc` a second time
(maybe because we regenerate/refresh `loaddefs.el` by error) after
Emacs was dumped, maybe during the "install" step that builds the
`Emacs.app`.

Normally, the way things are supposed to work is:

- build temacs.
- dump bootstrap-emacs using some old `loaddefs.el(c)` of
  `ldefs-boot.el`.
- compile the preloaded files and build the true `loaddefs.el`.
- dump the final `emacs` executable.
- keep compiling the rest.

But if "keep compiling the rest" ends up touching a dependency of
`loaddefs.el` subsequent steps like `install` could decide to make poor
decisions :-(

IOW rebuild, keeping a log and check what was done in which order.


        Stefan

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Eli Zaretskii]

> Date: Mon, 18 Sep 2023 17:25:20 +0200
> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Cc: monnier@iro.umontreal.ca, 66022@debbugs.gnu.org
> 
> On 23-09-18 17:18 , Gerd Möllmann wrote:
> > On 23-09-18 17:09 , Eli Zaretskii wrote:
> >>> From: Gerd Möllmann <gerd.moellmann@gmail.com> Cc:
> >>> monnier@iro.umontreal.ca,  66022@debbugs.gnu.org Date: Mon, 18 Sep
> >>> 2023 16:56:55 +0200
> >>>
> >>> Eli Zaretskii <eliz@gnu.org> writes:
> >>>
> >>>> Any idea why a call to 'documentation' loads loaddefs?  Did you
> >>>> indeed "hack the Emacs you were running"?
> >>>
> >>> No idea, and I didn't hack it at all.  This was a minute after I
> >>> started a newly-build Emacs, wondered why a C-c C-k binding didn't
> >>> work.  I then pressed C-c alone to see what it is bound to, with
> >>> which-key.
> >>
> >> So I think this now becomes the main question.  If this is not a
> >> bug, i.e. if loaddefs can legitimately be loaded mid-session, we
> >> cannot autoload key bindings, at least not naïvely as we do now.
> > 
> > Another observation: when I start emacs from its source directory
> > master/src, and not as bundle, i.e. by® clicking on Emacs.app in the
> > dock, this doesn't happen.
> > 
> > One difference in the two scenarios is that Emacs.app contains gzip'd el
> > files.
> 
> ~/emacs/master/ > ls -l 
> /Applications/Emacs.app/Contents/Resources/lisp/loaddefs.el*
> -rw-r--r--  1 gerd  admin   377687 Sep 17 16:36 
> /Applications/Emacs.app/Contents/Resources/lisp/loaddefs.el.gz
> -rw-r--r--  1 gerd  admin  1467107 Sep 17 16:38 
> /Applications/Emacs.app/Contents/Resource

Is your Emacs compiled with zlib?

> Note the timestamps.  Is that good or bad?

Good.  The .gz ffile should have the time stamp of the corresponding
.el file.

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Eli Zaretskii <eliz@gnu.org>,  66022@debbugs.gnu.org
> Date: Mon, 18 Sep 2023 12:23:31 -0400
> 
> >> Any idea why a call to 'documentation' loads loaddefs?  Did you indeed
> >> "hack the Emacs you were running"?
> > No idea, and I didn't hack it at all.  This was a minute after I started
> > a newly-build Emacs, wondered why a C-c C-k binding didn't work.
> 
> Hmm... sounds like we have a real bug, then!

Or a hidden feature ;-)

I think we should look at the conditions which will cause
'documentation' to load loaddefs.  Which part of it could load
loaddefs?

Maybe set a breakpoint in Fload conditioned by the FILE being
loaddefs.el(c), and look at the backtrace?

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Gerd Möllmann]

On 23-09-18 19:03 , Eli Zaretskii wrote:
>> Date: Mon, 18 Sep 2023 17:25:20 +0200
>> From: Gerd Möllmann <gerd.moellmann@gmail.com>
>> Cc: monnier@iro.umontreal.ca, 66022@debbugs.gnu.org
>>
>> On 23-09-18 17:18 , Gerd Möllmann wrote:
>>> On 23-09-18 17:09 , Eli Zaretskii wrote:
>>>>> From: Gerd Möllmann <gerd.moellmann@gmail.com> Cc:
>>>>> monnier@iro.umontreal.ca,  66022@debbugs.gnu.org Date: Mon, 18 Sep
>>>>> 2023 16:56:55 +0200
>>>>>
>>>>> Eli Zaretskii <eliz@gnu.org> writes:
>>>>>
>>>>>> Any idea why a call to 'documentation' loads loaddefs?  Did you
>>>>>> indeed "hack the Emacs you were running"?
>>>>>
>>>>> No idea, and I didn't hack it at all.  This was a minute after I
>>>>> started a newly-build Emacs, wondered why a C-c C-k binding didn't
>>>>> work.  I then pressed C-c alone to see what it is bound to, with
>>>>> which-key.
>>>>
>>>> So I think this now becomes the main question.  If this is not a
>>>> bug, i.e. if loaddefs can legitimately be loaded mid-session, we
>>>> cannot autoload key bindings, at least not naïvely as we do now.
>>>
>>> Another observation: when I start emacs from its source directory
>>> master/src, and not as bundle, i.e. by® clicking on Emacs.app in the
>>> dock, this doesn't happen.
>>>
>>> One difference in the two scenarios is that Emacs.app contains gzip'd el
>>> files.
>>
>> ~/emacs/master/ > ls -l
>> /Applications/Emacs.app/Contents/Resources/lisp/loaddefs.el*
>> -rw-r--r--  1 gerd  admin   377687 Sep 17 16:36
>> /Applications/Emacs.app/Contents/Resources/lisp/loaddefs.el.gz
>> -rw-r--r--  1 gerd  admin  1467107 Sep 17 16:38
>> /Applications/Emacs.app/Contents/Resource
> 
> Is your Emacs compiled with zlib?
> 
>> Note the timestamps.  Is that good or bad?

I think so.  config.log:

configure:25922: checking for library containing inflateEnd
configure:25972: result: -lz

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Gerd Möllmann]

On 23-09-18 19:00 , Stefan Monnier wrote:
>> Sorry, missed copying a line from the terminal
>> 
>> / >  ls -l 
>> /Applications/Emacs.app/Contents/Resources/lisp/loaddefs.el* 
>> -rw-r--r--  1 gerd  admin   377687 Sep 17 16:36 
>> /Applications/Emacs.app/Contents/Resources/lisp/loaddefs.el.gz 
>> -rw-r--r--  1 gerd  admin  1467107 Sep 17 16:38 
>> /Applications/Emacs.app/Contents/Resources/lisp/loaddefs.elc 
>> gerd@Pro 2023-09-18 18:32 ~/ >
> 
> I believe the `lread.c` code which calls `Fload` should be looking at
> the `loaddefs.elc` file here, so removing the `.el.gz` should make no
> difference.
> 
> Now why does it feel the need to reload this file?  The way the code
>  works is that it receives a (FILE . POS) pair where FILE should 
> presumably point to the `loaddefs.elc`.
> 
> When it opens that `.elc` file, POS should point to the beginning of 
> the desired docstring with nearby info indicating the length of that 
> docstring.
> 
> If the file was changed in the mean time, POS may end up pointing 
> elsewhere.  `lread.c` performs a sanity check to make sure POS
> points to something that does indeed look like a docstring and
> apparently in your case that sanity check fails.
> 
> My crystal ball suggests maybe you're witnessing a bug in the build 
> process where we end up *re*compiling `loaddefs.elc` a second time 
> (maybe because we regenerate/refresh `loaddefs.el` by error) after 
> Emacs was dumped, maybe during the "install" step that builds the 
> `Emacs.app`.
> 
> Normally, the way things are supposed to work is:
> 
> - build temacs. - dump bootstrap-emacs using some old 
> `loaddefs.el(c)` of `ldefs-boot.el`. - compile the preloaded files 
> and build the true `loaddefs.el`. - dump the final `emacs` 
> executable. - keep compiling the rest.
> 
> But if "keep compiling the rest" ends up touching a dependency of 
> `loaddefs.el` subsequent steps like `install` could decide to make 
> poor decisions :-(
> 
> IOW rebuild, keeping a log and check what was done in which order.

Ok, I'll try that during the next days.  (Have to finally replace the
battery of the old Macbook, and the display for the mini is not working.)

Do you think the following plan make ssense?

- I assume that a build from git clean -xdf shows all the steps that
must happen with 100% certainty.

- I'd then make bootstrap + look for a difference.

- Then same procedure for simple make in the toplevel dir,
    - after there is only a C file changed
    - after only a Lisp file is changed

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> Do you think the following plan make ssense?
>
> - I assume that a build from git clean -xdf shows all the steps that
> must happen with 100% certainty.
>
> - I'd then make bootstrap + look for a difference.
>
> - Then same procedure for simple make in the toplevel dir,
>    - after there is only a C file changed
>    - after only a Lisp file is changed

Sounds good.

You might also want to compare the timestamp of `loaddefs.elc` compared to
the timestamp of `emacs.pdmp` in your build directory (assuming you
still have it around).


        Stefan

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Gerd Möllmann]

On 23-09-18 20:24 , Stefan Monnier wrote:
>> Do you think the following plan make ssense?
>>
>> - I assume that a build from git clean -xdf shows all the steps that
>> must happen with 100% certainty.
>>
>> - I'd then make bootstrap + look for a difference.
>>
>> - Then same procedure for simple make in the toplevel dir,
>>     - after there is only a C file changed
>>     - after only a Lisp file is changed
> 
> Sounds good.
> 
> You might also want to compare the timestamp of `loaddefs.elc` compared to
> the timestamp of `emacs.pdmp` in your build directory (assuming you
> still have it around).

Thanks, will do.

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Gerd Möllmann]

On 23-09-18 20:24 , Stefan Monnier wrote:
>> Do you think the following plan make ssense?
>>
>> - I assume that a build from git clean -xdf shows all the steps that
>> must happen with 100% certainty.
>>
>> - I'd then make bootstrap + look for a difference.
>>
>> - Then same procedure for simple make in the toplevel dir,
>>     - after there is only a C file changed
>>     - after only a Lisp file is changed
> 
> Sounds good.
> 
> You might also want to compare the timestamp of `loaddefs.elc` compared to
> the timestamp of `emacs.pdmp` in your build directory (assuming you
> still have it around).

I think I've found something.

Assume I git pull, and emacs-lisp/comp.el is modified in a way that 
loaddefs gets regenerated and written to disk.  Then do a toplevel 
gmake.  After that, I see the following:

~/emacs/master/ > ll lisp/loaddefs.el* src/emacs.pdmp 
lisp/emacs-lisp/comp.* 
nextstep/Emacs.app/Contents/Resources/lisp/loaddefs* 
nextstep/Emacs.app/Contents/MacOS/libexec/Emacs.pdmp
-rw-r--r--  1 gerd  staff   1.4M Sep 20 11:31 lisp/loaddefs.el
-rw-r--r--  1 gerd  staff   1.4M Sep 20 11:32 lisp/loaddefs.elc
-rw-r--r--  2 gerd  staff    16M Sep 20 11:32 src/emacs.pdmp
-rw-r--r--  1 gerd  staff   188K Sep 20 11:31 lisp/emacs-lisp/comp.el
-rw-r--r--  1 gerd  staff   399K Sep 20 11:32 lisp/emacs-lisp/comp.elc
-rw-r--r--  1 gerd  staff   369K Sep 20 10:55 
nextstep/Emacs.app/Contents/Resources/lisp/loaddefs.el.gz
-rw-r--r--  1 gerd  staff   1.4M Sep 20 10:56 
nextstep/Emacs.app/Contents/Resources/lisp/loaddefs.elc
-rw-r--r--  1 gerd  staff    16M Sep 20 11:32 
nextstep/Emacs.app/Contents/MacOS/libexec/Emacs.pdmp

Note that the pdmp in Emacs.app is new, while the loaddefs under 
nextstep are old.

After gmake install:

~/emacs/master/ > ll lisp/loaddefs.el* src/emacs.pdmp 
lisp/emacs-lisp/comp.* 
nextstep/Emacs.app/Contents/Resources/lisp/loaddefs* 
nextstep/Emacs.app/Contents/MacOS/libexec/Emacs.pdmp
-rw-r--r--  1 gerd  staff   1.4M Sep 20 11:31 lisp/loaddefs.el
-rw-r--r--  1 gerd  staff   1.4M Sep 20 11:32 lisp/loaddefs.elc
-rw-r--r--  2 gerd  staff    16M Sep 20 11:32 src/emacs.pdmp
-rw-r--r--  1 gerd  staff   188K Sep 20 11:31 lisp/emacs-lisp/comp.el
-rw-r--r--  1 gerd  staff   399K Sep 20 11:32 lisp/emacs-lisp/comp.elc
-rw-r--r--  1 gerd  staff   369K Sep 20 11:31 
nextstep/Emacs.app/Contents/Resources/lisp/loaddefs.el.gz
-rw-r--r--  1 gerd  staff   1.4M Sep 20 11:32 
nextstep/Emacs.app/Contents/Resources/lisp/loaddefs.elc
-rw-r--r--  1 gerd  staff    16M Sep 20 11:32 
nextstep/Emacs.app/Contents/MacOS/libexec/Emacs.pdmp

Means to me that if I just gmake, expecting that could not possibly 
change Emacs.app, I'm quite mistaken.  Instead, I now have an 
inconsistent Emacs.app.

Does that make sense?

Has someone maybe an idea why the pdmp gets installed so early?

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Eli Zaretskii]

> Date: Wed, 20 Sep 2023 11:57:28 +0200
> Cc: Eli Zaretskii <eliz@gnu.org>, 66022@debbugs.gnu.org
> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> 
> Assume I git pull, and emacs-lisp/comp.el is modified in a way that 
> loaddefs gets regenerated and written to disk.  Then do a toplevel 
> gmake.  After that, I see the following:
> 
> ~/emacs/master/ > ll lisp/loaddefs.el* src/emacs.pdmp 
> lisp/emacs-lisp/comp.* 
> nextstep/Emacs.app/Contents/Resources/lisp/loaddefs* 
> nextstep/Emacs.app/Contents/MacOS/libexec/Emacs.pdmp
> -rw-r--r--  1 gerd  staff   1.4M Sep 20 11:31 lisp/loaddefs.el
> -rw-r--r--  1 gerd  staff   1.4M Sep 20 11:32 lisp/loaddefs.elc
> -rw-r--r--  2 gerd  staff    16M Sep 20 11:32 src/emacs.pdmp
> -rw-r--r--  1 gerd  staff   188K Sep 20 11:31 lisp/emacs-lisp/comp.el
> -rw-r--r--  1 gerd  staff   399K Sep 20 11:32 lisp/emacs-lisp/comp.elc
> -rw-r--r--  1 gerd  staff   369K Sep 20 10:55 
> nextstep/Emacs.app/Contents/Resources/lisp/loaddefs.el.gz
> -rw-r--r--  1 gerd  staff   1.4M Sep 20 10:56 
> nextstep/Emacs.app/Contents/Resources/lisp/loaddefs.elc
> -rw-r--r--  1 gerd  staff    16M Sep 20 11:32 
> nextstep/Emacs.app/Contents/MacOS/libexec/Emacs.pdmp
> 
> Note that the pdmp in Emacs.app is new, while the loaddefs under 
> nextstep are old.
> 
> After gmake install:
> 
> ~/emacs/master/ > ll lisp/loaddefs.el* src/emacs.pdmp 
> lisp/emacs-lisp/comp.* 
> nextstep/Emacs.app/Contents/Resources/lisp/loaddefs* 
> nextstep/Emacs.app/Contents/MacOS/libexec/Emacs.pdmp
> -rw-r--r--  1 gerd  staff   1.4M Sep 20 11:31 lisp/loaddefs.el
> -rw-r--r--  1 gerd  staff   1.4M Sep 20 11:32 lisp/loaddefs.elc
> -rw-r--r--  2 gerd  staff    16M Sep 20 11:32 src/emacs.pdmp
> -rw-r--r--  1 gerd  staff   188K Sep 20 11:31 lisp/emacs-lisp/comp.el
> -rw-r--r--  1 gerd  staff   399K Sep 20 11:32 lisp/emacs-lisp/comp.elc
> -rw-r--r--  1 gerd  staff   369K Sep 20 11:31 
> nextstep/Emacs.app/Contents/Resources/lisp/loaddefs.el.gz
> -rw-r--r--  1 gerd  staff   1.4M Sep 20 11:32 
> nextstep/Emacs.app/Contents/Resources/lisp/loaddefs.elc
> -rw-r--r--  1 gerd  staff    16M Sep 20 11:32 
> nextstep/Emacs.app/Contents/MacOS/libexec/Emacs.pdmp
> 
> Means to me that if I just gmake, expecting that could not possibly 
> change Emacs.app, I'm quite mistaken.  Instead, I now have an 
> inconsistent Emacs.app.
> 
> Does that make sense?
> 
> Has someone maybe an idea why the pdmp gets installed so early?

You are saying that building Emacs in the source tree somehow updates
the .pdmp file in the _installation_ tree?  That's got to be
macOS-specific, because that should never happen.  The installation
tree is modified only by "make install".

Any idea which place in the Makefile's or in loadup.el does that?

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Gerd Möllmann]

On 23-09-20 16:11 , Eli Zaretskii wrote:
>> Date: Wed, 20 Sep 2023 11:57:28 +0200
>> Cc: Eli Zaretskii <eliz@gnu.org>, 66022@debbugs.gnu.org
>> From: Gerd Möllmann <gerd.moellmann@gmail.com>
>>
>> Assume I git pull, and emacs-lisp/comp.el is modified in a way that
>> loaddefs gets regenerated and written to disk.  Then do a toplevel
>> gmake.  After that, I see the following:
>>
>> ~/emacs/master/ > ll lisp/loaddefs.el* src/emacs.pdmp
>> lisp/emacs-lisp/comp.*
>> nextstep/Emacs.app/Contents/Resources/lisp/loaddefs*
>> nextstep/Emacs.app/Contents/MacOS/libexec/Emacs.pdmp
>> -rw-r--r--  1 gerd  staff   1.4M Sep 20 11:31 lisp/loaddefs.el
>> -rw-r--r--  1 gerd  staff   1.4M Sep 20 11:32 lisp/loaddefs.elc
>> -rw-r--r--  2 gerd  staff    16M Sep 20 11:32 src/emacs.pdmp
>> -rw-r--r--  1 gerd  staff   188K Sep 20 11:31 lisp/emacs-lisp/comp.el
>> -rw-r--r--  1 gerd  staff   399K Sep 20 11:32 lisp/emacs-lisp/comp.elc
>> -rw-r--r--  1 gerd  staff   369K Sep 20 10:55
>> nextstep/Emacs.app/Contents/Resources/lisp/loaddefs.el.gz
>> -rw-r--r--  1 gerd  staff   1.4M Sep 20 10:56
>> nextstep/Emacs.app/Contents/Resources/lisp/loaddefs.elc
>> -rw-r--r--  1 gerd  staff    16M Sep 20 11:32
>> nextstep/Emacs.app/Contents/MacOS/libexec/Emacs.pdmp
>>
>> Note that the pdmp in Emacs.app is new, while the loaddefs under
>> nextstep are old.
>>
>> After gmake install:
>>
>> ~/emacs/master/ > ll lisp/loaddefs.el* src/emacs.pdmp
>> lisp/emacs-lisp/comp.*
>> nextstep/Emacs.app/Contents/Resources/lisp/loaddefs*
>> nextstep/Emacs.app/Contents/MacOS/libexec/Emacs.pdmp
>> -rw-r--r--  1 gerd  staff   1.4M Sep 20 11:31 lisp/loaddefs.el
>> -rw-r--r--  1 gerd  staff   1.4M Sep 20 11:32 lisp/loaddefs.elc
>> -rw-r--r--  2 gerd  staff    16M Sep 20 11:32 src/emacs.pdmp
>> -rw-r--r--  1 gerd  staff   188K Sep 20 11:31 lisp/emacs-lisp/comp.el
>> -rw-r--r--  1 gerd  staff   399K Sep 20 11:32 lisp/emacs-lisp/comp.elc
>> -rw-r--r--  1 gerd  staff   369K Sep 20 11:31
>> nextstep/Emacs.app/Contents/Resources/lisp/loaddefs.el.gz
>> -rw-r--r--  1 gerd  staff   1.4M Sep 20 11:32
>> nextstep/Emacs.app/Contents/Resources/lisp/loaddefs.elc
>> -rw-r--r--  1 gerd  staff    16M Sep 20 11:32
>> nextstep/Emacs.app/Contents/MacOS/libexec/Emacs.pdmp
>>
>> Means to me that if I just gmake, expecting that could not possibly
>> change Emacs.app, I'm quite mistaken.  Instead, I now have an
>> inconsistent Emacs.app.
>>
>> Does that make sense?
>>
>> Has someone maybe an idea why the pdmp gets installed so early?
> 
> You are saying that building Emacs in the source tree somehow updates
> the .pdmp file in the _installation_ tree?  That's got to be
> macOS-specific, because that should never happen.  The installation
> tree is modified only by "make install".

Yup it is.  The installation is done in src/nextstep/Emacs.app/. 
Emacs.app is a macOS bundle that can then be copied/moved to where the 
user wants, for example to /Applications.

> 
> Any idea which place in the Makefile's or in loadup.el does that?

I think nextstep/Makefile.in contains

# FIXME: Don't install the dump file into the app bundle when
# self-contained install is disabled.
${ns_applibexecdir}/Emacs.pdmp: ${ns_appdir} ${ns_check_file} 
../src/emacs${EXEEXT}.pdmp
	${MKDIR_P} ${ns_applibexecdir}
	cp -f ../src/emacs${EXEEXT}.pdmp $@

and the "all" target has that the pdmp as a dependency.

I have no idea though what the comment means, or if the cp if necessary 
for something.

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Eli Zaretskii]

> Date: Wed, 20 Sep 2023 16:22:24 +0200
> Cc: monnier@iro.umontreal.ca, 66022@debbugs.gnu.org
> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> 
> > You are saying that building Emacs in the source tree somehow updates
> > the .pdmp file in the _installation_ tree?  That's got to be
> > macOS-specific, because that should never happen.  The installation
> > tree is modified only by "make install".
> 
> Yup it is.  The installation is done in src/nextstep/Emacs.app/. 
> Emacs.app is a macOS bundle that can then be copied/moved to where the 
> user wants, for example to /Applications.
> 
> > 
> > Any idea which place in the Makefile's or in loadup.el does that?
> 
> I think nextstep/Makefile.in contains
> 
> # FIXME: Don't install the dump file into the app bundle when
> # self-contained install is disabled.
> ${ns_applibexecdir}/Emacs.pdmp: ${ns_appdir} ${ns_check_file} 
> ../src/emacs${EXEEXT}.pdmp
> 	${MKDIR_P} ${ns_applibexecdir}
> 	cp -f ../src/emacs${EXEEXT}.pdmp $@
> 
> and the "all" target has that the pdmp as a dependency.
> 
> I have no idea though what the comment means, or if the cp if necessary 
> for something.

I guess no one imagined that someone will build Emacs from Git _and_
install it.  I' for example, run it directly from the source tree.

But anyway, all you should remember is to say "make install" each time
you rebuild, and then this problem will not happen.  Right?

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Gerd Möllmann]

On 23-09-20 17:04 , Eli Zaretskii wrote:
>> Date: Wed, 20 Sep 2023 16:22:24 +0200
>> Cc: monnier@iro.umontreal.ca, 66022@debbugs.gnu.org
>> From: Gerd Möllmann <gerd.moellmann@gmail.com>
>>
>>> You are saying that building Emacs in the source tree somehow updates
>>> the .pdmp file in the _installation_ tree?  That's got to be
>>> macOS-specific, because that should never happen.  The installation
>>> tree is modified only by "make install".
>>
>> Yup it is.  The installation is done in src/nextstep/Emacs.app/.
>> Emacs.app is a macOS bundle that can then be copied/moved to where the
>> user wants, for example to /Applications.
>>
>>>
>>> Any idea which place in the Makefile's or in loadup.el does that?
>>
>> I think nextstep/Makefile.in contains
>>
>> # FIXME: Don't install the dump file into the app bundle when
>> # self-contained install is disabled.
>> ${ns_applibexecdir}/Emacs.pdmp: ${ns_appdir} ${ns_check_file}
>> ../src/emacs${EXEEXT}.pdmp
>> 	${MKDIR_P} ${ns_applibexecdir}
>> 	cp -f ../src/emacs${EXEEXT}.pdmp $@
>>
>> and the "all" target has that the pdmp as a dependency.
>>
>> I have no idea though what the comment means, or if the cp if necessary
>> for something.
> 
> I guess no one imagined that someone will build Emacs from Git _and_
> install it.  I' for example, run it directly from the source tree.
> 
> But anyway, all you should remember is to say "make install" each time
> you rebuild, and then this problem will not happen.  Right?

Well, the reason I do this to make sure I have a "good" build available, 
in case something.  But that's not a big problem, once one knows that a 
gmake (without install) does what it does :-).

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> Any idea which place in the Makefile's or in loadup.el does that?
>
> I think nextstep/Makefile.in contains
>
> # FIXME: Don't install the dump file into the app bundle when
> # self-contained install is disabled.
> ${ns_applibexecdir}/Emacs.pdmp: ${ns_appdir} ${ns_check_file}
> ../src/emacs${EXEEXT}.pdmp
> 	${MKDIR_P} ${ns_applibexecdir}
> 	cp -f ../src/emacs${EXEEXT}.pdmp $@
>
> and the "all" target has that the pdmp as a dependency.

Sounds like a bug, yes: either the same thing should be done for all the
files in ${ns_applibexecdir} or it should be done for none.

This said, I'm, 100% unfamiliar with the way the `Emacs.app` directory
is built, so maybe this is a feature rather than a bug.

> I have no idea though what the comment means, or if the cp if necessary
> for something.

The FIXME sounds to me like it's hinting at the same bug you're seeing
(tho maybe another manifestation of it), and since there's a FIXME
I presume that fixing it isn't as simple as removing this rule :-)


        Stefan

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Gerd Möllmann]

On 23-09-20 18:56 , Stefan Monnier wrote:
>>> Any idea which place in the Makefile's or in loadup.el does that?
>>
>> I think nextstep/Makefile.in contains
>>
>> # FIXME: Don't install the dump file into the app bundle when
>> # self-contained install is disabled.
>> ${ns_applibexecdir}/Emacs.pdmp: ${ns_appdir} ${ns_check_file}
>> ../src/emacs${EXEEXT}.pdmp
>> 	${MKDIR_P} ${ns_applibexecdir}
>> 	cp -f ../src/emacs${EXEEXT}.pdmp $@
>>
>> and the "all" target has that the pdmp as a dependency.
> 
> Sounds like a bug, yes: either the same thing should be done for all the
> files in ${ns_applibexecdir} or it should be done for none.
> 
> This said, I'm, 100% unfamiliar with the way the `Emacs.app` directory
> is built, so maybe this is a feature rather than a bug.
> 
>> I have no idea though what the comment means, or if the cp if necessary
>> for something.
> 
> The FIXME sounds to me like it's hinting at the same bug you're seeing
> (tho maybe another manifestation of it), and since there's a FIXME
> I presume that fixing it isn't as simple as removing this rule :-)

Hi Alan,

this bug finally ended with finding that a plain "gmake" in the toplevel 
Emacs tree can copy src/emacs.pdmp to nextstep/Emacs.app/... which lead 
to me being completely puzzled.

Do you perhaps remember what the FIXME in nextstep/Makefile.in is about 
(see above)?  I guess a self-contained build is one building Emacs.app? 
I don't know what kind of build it is when it is not self-contained.

The FIXME comes from this commit.

5dd2d50f3d5e65b85c87da86e2e8a6d087fe5767
Author:     Alan Third <alan@idiocy.org>
AuthorDate: Wed Jun 16 21:28:10 2021 +0100
Commit:     Alan Third <alan@idiocy.org>
CommitDate: Sat Jun 26 10:34:59 2021 +0100

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Alan Third]

On Wed, Sep 20, 2023 at 08:24:52PM +0200, Gerd Möllmann wrote:
> On 23-09-20 18:56 , Stefan Monnier wrote:
> > > > Any idea which place in the Makefile's or in loadup.el does that?
> > > 
> > > I think nextstep/Makefile.in contains
> > > 
> > > # FIXME: Don't install the dump file into the app bundle when
> > > # self-contained install is disabled.
> > > ${ns_applibexecdir}/Emacs.pdmp: ${ns_appdir} ${ns_check_file}
> > > ../src/emacs${EXEEXT}.pdmp
> > > 	${MKDIR_P} ${ns_applibexecdir}
> > > 	cp -f ../src/emacs${EXEEXT}.pdmp $@
> > > 
> > > and the "all" target has that the pdmp as a dependency.
> > 
> > Sounds like a bug, yes: either the same thing should be done for all the
> > files in ${ns_applibexecdir} or it should be done for none.
> > 
> > This said, I'm, 100% unfamiliar with the way the `Emacs.app` directory
> > is built, so maybe this is a feature rather than a bug.
> > 
> > > I have no idea though what the comment means, or if the cp if necessary
> > > for something.
> > 
> > The FIXME sounds to me like it's hinting at the same bug you're seeing
> > (tho maybe another manifestation of it), and since there's a FIXME
> > I presume that fixing it isn't as simple as removing this rule :-)
> 
> Hi Alan,
> 
> this bug finally ended with finding that a plain "gmake" in the toplevel
> Emacs tree can copy src/emacs.pdmp to nextstep/Emacs.app/... which lead to
> me being completely puzzled.
> 
> Do you perhaps remember what the FIXME in nextstep/Makefile.in is about (see
> above)?  I guess a self-contained build is one building Emacs.app? I don't
> know what kind of build it is when it is not self-contained.

Yeah. A non-self contained build is a standard UNIX style install.
i.e. the files go into /usr/local or whatever --prefix is set to.

The slight fly in the ointment here is that we support the ability to
do that normal UNIX-style install, but still have an Emacs.app that
will run the installed Emacs. I don't really understand how it works,
or really why anyone would want to do that, but it's described in the
nextstep readme, iirc, and people do sometimes use that functionality.

I put that FIXME in because I couldn't work out how NOT to install the
pdmp file into the .app dir when doing that UNIX style install. It
still gets installed somewhere under PREFIX, but is copied into the
.app as well, even though that copy is never used.

I'm not great at writing Makefiles.
-- 
Alan Third

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Gerd Möllmann]

On 23-09-20 20:50 , Alan Third wrote:
> On Wed, Sep 20, 2023 at 08:24:52PM +0200, Gerd Möllmann wrote:
>> On 23-09-20 18:56 , Stefan Monnier wrote:
>>>>> Any idea which place in the Makefile's or in loadup.el does that?
>>>>
>>>> I think nextstep/Makefile.in contains
>>>>
>>>> # FIXME: Don't install the dump file into the app bundle when
>>>> # self-contained install is disabled.
>>>> ${ns_applibexecdir}/Emacs.pdmp: ${ns_appdir} ${ns_check_file}
>>>> ../src/emacs${EXEEXT}.pdmp
>>>> 	${MKDIR_P} ${ns_applibexecdir}
>>>> 	cp -f ../src/emacs${EXEEXT}.pdmp $@
>>>>
>>>> and the "all" target has that the pdmp as a dependency.
>>>
>>> Sounds like a bug, yes: either the same thing should be done for all the
>>> files in ${ns_applibexecdir} or it should be done for none.
>>>
>>> This said, I'm, 100% unfamiliar with the way the `Emacs.app` directory
>>> is built, so maybe this is a feature rather than a bug.
>>>
>>>> I have no idea though what the comment means, or if the cp if necessary
>>>> for something.
>>>
>>> The FIXME sounds to me like it's hinting at the same bug you're seeing
>>> (tho maybe another manifestation of it), and since there's a FIXME
>>> I presume that fixing it isn't as simple as removing this rule :-)
>>
>> Hi Alan,
>>
>> this bug finally ended with finding that a plain "gmake" in the toplevel
>> Emacs tree can copy src/emacs.pdmp to nextstep/Emacs.app/... which lead to
>> me being completely puzzled.
>>
>> Do you perhaps remember what the FIXME in nextstep/Makefile.in is about (see
>> above)?  I guess a self-contained build is one building Emacs.app? I don't
>> know what kind of build it is when it is not self-contained.
> 
> Yeah. A non-self contained build is a standard UNIX style install.
> i.e. the files go into /usr/local or whatever --prefix is set to.
> 
> The slight fly in the ointment here is that we support the ability to
> do that normal UNIX-style install, but still have an Emacs.app that
> will run the installed Emacs. I don't really understand how it works,
> or really why anyone would want to do that, but it's described in the
> nextstep readme, iirc, and people do sometimes use that functionality.
> 
> I put that FIXME in because I couldn't work out how NOT to install the
> pdmp file into the .app dir when doing that UNIX style install. It
> still gets installed somewhere under PREFIX, but is copied into the
> .app as well, even though that copy is never used.
> 
> I'm not great at writing Makefiles.

Thanks for the explanation, Alan!  I'd never had expected that there is 
an installation to /usr/local on a Mac.  But maybe Nextstep does it that 
way, never used that.

Hm, maybe I can try something.  I think I've seen a config variable 
somewhere.  I'll come back tomorrow probably, it's a bit late now.

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Alan Third]

On Wed, Sep 20, 2023 at 12:56:22PM -0400, Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors wrote:
> >> Any idea which place in the Makefile's or in loadup.el does that?
> >
> > I think nextstep/Makefile.in contains
> >
> > # FIXME: Don't install the dump file into the app bundle when
> > # self-contained install is disabled.
> > ${ns_applibexecdir}/Emacs.pdmp: ${ns_appdir} ${ns_check_file}
> > ../src/emacs${EXEEXT}.pdmp
> > 	${MKDIR_P} ${ns_applibexecdir}
> > 	cp -f ../src/emacs${EXEEXT}.pdmp $@
> >
> > and the "all" target has that the pdmp as a dependency.
> 
> Sounds like a bug, yes: either the same thing should be done for all the
> files in ${ns_applibexecdir} or it should be done for none.
> 
> This said, I'm, 100% unfamiliar with the way the `Emacs.app` directory
> is built, so maybe this is a feature rather than a bug.

I think it's probably a bug. IIRC the Emacs executable in the
Emacs.app directory is updated when you do a "make", and the above
code runs at the same time, otherwise you end up with an inconsistent
.app.

Clearly you get an inconsistent .app anyway.

I don't know why it works that way, I was never able to fully
understand how Emacs.app is built, and I guess someone either thought
it was a good idea or made a mistake.

-- 
Alan Third

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Gerd Möllmann]

On 23-09-20 20:50 , Alan Third wrote:
> On Wed, Sep 20, 2023 at 08:24:52PM +0200, Gerd Möllmann wrote:
>> On 23-09-20 18:56 , Stefan Monnier wrote:
>>>>> Any idea which place in the Makefile's or in loadup.el does that?
>>>>
>>>> I think nextstep/Makefile.in contains
>>>>
>>>> # FIXME: Don't install the dump file into the app bundle when
>>>> # self-contained install is disabled.
>>>> ${ns_applibexecdir}/Emacs.pdmp: ${ns_appdir} ${ns_check_file}
>>>> ../src/emacs${EXEEXT}.pdmp
>>>> 	${MKDIR_P} ${ns_applibexecdir}
>>>> 	cp -f ../src/emacs${EXEEXT}.pdmp $@
>>>>
>>>> and the "all" target has that the pdmp as a dependency.
>>>
>>> Sounds like a bug, yes: either the same thing should be done for all the
>>> files in ${ns_applibexecdir} or it should be done for none.
>>>
>>> This said, I'm, 100% unfamiliar with the way the `Emacs.app` directory
>>> is built, so maybe this is a feature rather than a bug.
>>>
>>>> I have no idea though what the comment means, or if the cp if necessary
>>>> for something.
>>>
>>> The FIXME sounds to me like it's hinting at the same bug you're seeing
>>> (tho maybe another manifestation of it), and since there's a FIXME
>>> I presume that fixing it isn't as simple as removing this rule :-)
>>
>> Hi Alan,
>>
>> this bug finally ended with finding that a plain "gmake" in the toplevel
>> Emacs tree can copy src/emacs.pdmp to nextstep/Emacs.app/... which lead to
>> me being completely puzzled.
>>
>> Do you perhaps remember what the FIXME in nextstep/Makefile.in is about (see
>> above)?  I guess a self-contained build is one building Emacs.app? I don't
>> know what kind of build it is when it is not self-contained.
> 
> Yeah. A non-self contained build is a standard UNIX style install.
> i.e. the files go into /usr/local or whatever --prefix is set to.
> 
> The slight fly in the ointment here is that we support the ability to
> do that normal UNIX-style install, but still have an Emacs.app that
> will run the installed Emacs. I don't really understand how it works,
> or really why anyone would want to do that, but it's described in the
> nextstep readme, iirc, and people do sometimes use that functionality.
> 
> I put that FIXME in because I couldn't work out how NOT to install the
> pdmp file into the .app dir when doing that UNIX style install. It
> still gets installed somewhere under PREFIX, but is copied into the
> .app as well, even though that copy is never used.
> 
> I'm not great at writing Makefiles.

I think I fixed this now on master, in 
d0b1e3647fb4e6d889f9f999388c53daf0e67f0d.

This leaves a non-self-contained alone, just in case.

bug#66022: 30.0.50; kmacro overwriting global keybindings

[Gerd Möllmann]

Fixed for me with d0b1e3647fb, so I'm closing this.