bug#68083: 30.0.50; Intermittent build failure with native compilation

bug#68083: 30.0.50; Intermittent build failure with native compilation

[Aaron Jensen]

On macOS, I am often getting this when building Emacs from scratch. Could this be a race condition with a parallel build?

I bisected and the problematic commit is: e670412a3e101e70dc26e021f467faece8cb7f6b

In toplevel form:
org/org-element.el:64:2: Error: Eager macro-expansion failure: (native-compiler-error (lambda (arg322 &optional arg323) (let ((f #'macroexpand)) (funcall f arg322 arg323))) "Compiling /private/var/tmp/emacs-plusA30-20231227-
10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln...
File already exists: /private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln

Error: file-already-exists (\"File already exists\" \"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln\")
  mapbacktrace(#f(compiled-function (evald func args flags) #<bytecode -0x1fcb3a734512f81>))
  debug-early-backtrace()
  debug-early(error (file-already-exists \"File already exists\" \"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln
\"))
  rename-file(\"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_08i0tl8.eln.tmp\" \"/private/var/tmp/emacs-plusA30-20231
227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln\")
  comp-delete-or-replace-file(\"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln\" \"/private/var/tmp/emacs-plusA30
-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_08i0tl8.eln.tmp\")
  comp--compile-ctxt-to-file(\"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln\")
  comp-compile-ctxt-to-file(\"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln\")
  comp-final1()
  load-with-code-conversion(\"/private/var/tmp/emacs-int-comp-subr--trampoline-6d6163726f657870616e64_macroexpand_0-haajZW.el\" \"/private/var/tmp/emacs-int-comp-subr--trampoline-6d6163726f657870616e64_macroexpand_0-haajZW.e
l\" nil t)
  command-line-1((\"-l\" \"/private/var/tmp/emacs-int-comp-subr--trampoline-6d6163726f657870616e64_macroexpand_0-haajZW.el\"))
  command-line()
  normal-top-level()
")
gmake[3]: *** [Makefile:330: org/org-element.elc] Error 1








In GNU Emacs 30.0.50 (build 1, aarch64-apple-darwin23.2.0, NS
 appkit-2487.30 Version 14.2.1 (Build 23C71)) of 2023-12-27 built on
 Aarons-MacBook-Pro.local
Windowing system distributor 'Apple', version 10.3.2487
System Description:  macOS 14.2.1

Configured using:
 'configure --disable-dependency-tracking --disable-silent-rules
 --enable-locallisppath=/opt/homebrew/share/emacs/site-lisp
 --infodir=/opt/homebrew/Cellar/emacs-plus@30/30.0.50/share/info/emacs
 --prefix=/opt/homebrew/Cellar/emacs-plus@30/30.0.50 --with-xml2
 --with-gnutls --with-native-compilation --without-compress-install
 --without-dbus --without-imagemagick --with-modules --with-rsvg
 --with-webp --with-ns --disable-ns-self-contained 'CFLAGS=-Os -w -pipe
 -mmacosx-version-min=14
 -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk
 -DFD_SETSIZE=10000 -DDARWIN_UNLIMITED_SELECT'
 'CPPFLAGS=-I/opt/homebrew/opt/zlib/include
 -I/opt/homebrew/opt/jpeg/include -I/opt/homebrew/opt/icu4c/include
 -isystem/opt/homebrew/include -F/opt/homebrew/Frameworks
 -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk'
 'LDFLAGS=-L/opt/homebrew/opt/zlib/lib -L/opt/homebrew/opt/jpeg/lib
 -L/opt/homebrew/opt/icu4c/lib -L/opt/homebrew/lib
 -F/opt/homebrew/Frameworks -Wl,-headerpad_max_install_names
 -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk
 -L/opt/homebrew/opt/libgccjit/lib''

Configured features:
ACL GIF GLIB GMP GNUTLS JPEG JSON LCMS2 LIBXML2 MODULES NATIVE_COMP
NOTIFY KQUEUE NS PDUMPER PNG RSVG SQLITE3 THREADS TIFF
TOOLKIT_SCROLL_BARS TREE_SITTER WEBP XIM ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Text

Minor modes in effect:
  windmove-mode: t
  global-evil-mc-mode: t
  evil-mc-mode: t
  global-flycheck-mode: t
  flycheck-mode: t
  global-git-commit-mode: t
  transient-posframe-mode: t
  corfu-prescient-mode: t
  corfu-history-mode: t
  eval-sexp-fu-flash-mode: t
  tabspaces-mode: t
  org-roam-db-autosync-mode: t
  yas-global-mode: t
  yas-minor-mode: t
  vertico-prescient-mode: t
  prescient-persist-mode: t
  vertico-mouse-mode: t
  vertico-mode: t
  mini-frame-mode: t
  better-jumper-mode: t
  better-jumper-local-mode: t
  ns-auto-titlebar-mode: t
  global-anzu-mode: t
  anzu-mode: t
  which-key-posframe-mode: t
  which-key-mode: t
  gcmh-mode: t
  xterm-mouse-mode: t
  global-auto-revert-mode: t
  save-place-mode: t
  winner-mode: t
  savehist-mode: t
  delete-selection-mode: t
  recentf-mode: t
  repeat-mode: t
  +popup-mode: t
  evil-mode: t
  evil-local-mode: t
  server-mode: t
  leader-key-leader-override-mode: t
  global-leader-key-leader-override-mode: t
  elpaca-use-package-mode: t
  override-global-mode: t
  global-display-line-numbers-mode: t
  display-line-numbers-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tab-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  window-divider-mode: t
  minibuffer-regexp-mode: t
  line-number-mode: t
  auto-fill-function: #[128 \304\300\301%3#\207 [yas--auto-fill do-auto-fill :around nil apply] 5 advice]
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
/Users/aaronjensen/.emacs.d/elpaca/builds/lispy/elpa hides /Users/aaronjensen/.emacs.d/elpaca/builds/ivy/elpa

Features:
(org-clock image-file image-converter org-drill persist tramp-cmds
dabbrev tab-line evil-collection-vterm vterm tramp tramp-message
trampver tramp-integration files-x tramp-compat tramp-loaddefs term
ehelp vterm-module term/xterm xterm json-mode json-snatcher lua-mode
lsp-jq lsp-zig lsp-tilt lsp-steep lsp-svelte lsp-sqls lsp-solidity
lsp-ruby-syntax-tree lsp-ruby-lsp lsp-yaml lsp-xml lsp-vimscript
lsp-vhdl lsp-volar lsp-vetur lsp-html lsp-verilog lsp-vala lsp-v
lsp-typeprof lsp-ttcn3 lsp-toml lsp-terraform lsp-tex lsp-sorbet
lsp-solargraph lsp-semgrep lsp-rust lsp-rubocop lsp-rf lsp-ruff-lsp
lsp-remark lsp-racket lsp-r lsp-purescript lsp-pylsp lsp-pyls lsp-pwsh
lsp-php lsp-pls lsp-perlnavigator lsp-perl lsp-openscad lsp-ocaml
lsp-mojo lsp-magik lsp-nix lsp-nim lsp-nginx lsp-move lsp-mint lsp-mdx
lsp-marksman lsp-markdown lsp-lua lsp-kotlin lsp-json lsp-javascript
lsp-idris lsp-haxe lsp-groovy lsp-hack lsp-graphql lsp-glsl lsp-gleam
lsp-go lsp-completion lsp-gdscript lsp-fsharp lsp-fortran lsp-eslint
lsp-erlang lsp-emmet lsp-elixir lsp-elm lsp-dockerfile lsp-dhall lsp-d
lsp-cypher lsp-css lsp-csharp lsp-crystal lsp-credo lsp-cmake
lsp-clojure lsp-semantic-tokens lsp-clangd lsp-beancount lsp-bash
lsp-astro lsp-awk lsp-ansible lsp-angular lsp-ada lsp-actionscript
evil-collection-simple lsp-mode lsp-protocol spinner markdown-mode
inline ht js c-ts-common cc-mode cc-fonts cc-guess cc-menus cc-cmds
cc-styles cc-align cc-engine sql view cal-iso diary-lib diary-loaddefs
evil-ruby-text-objects ruby-refactor bundler inf-ruby ruby-mode
sh-script smie treesit epa-file network-stream mailalias smtpmail
textsec uni-scripts idna-mapping uni-confusable textsec-check consult
magit-bookmark bookmark dired-aux shadow sort mail-extr emacsbug
evil-matchit-simple evil-matchit-prog evil-matchit evil-matchit-sdk
semantic/lex semantic/fw hippie-exp evil-collection-helpful helpful
cc-langs cc-vars cc-defs trace info-look elisp-refs hide-mode-line info
magit-patch magit-subtree magit-gitignore magit-ediff
evil-collection-ediff ediff ediff-merg ediff-mult ediff-wind ediff-diff
ediff-help ediff-init ediff-util magit-extras vc-hg vc-bzr vc-src
vc-sccs vc-svn vc-cvs vc-rcs log-view bug-reference elpaca-log elpaca-ui
popup-mode-core windmove evil-terminal-cursor-changer color executable
evil-mc evil-mc-command-execute evil-mc-command-record
evil-mc-cursor-make evil-mc-region evil-mc-cursor-state evil-mc-undo
evil-mc-vars evil-mc-known-commands evil-mc-common magit-delta
xterm-color flycheck evil-collection-magit magit-submodule magit-blame
magit-stash magit-reflog magit-bisect magit-push magit-pull magit-fetch
magit-clone magit-remote magit-commit magit-sequence magit-notes
magit-worktree magit-tag magit-merge magit-branch magit-reset
magit-files magit-refs magit-status magit package url-handlers
magit-repos magit-apply magit-wip magit-log which-func magit-diff
smerge-mode diff git-commit log-edit pcvs-util add-log magit-core
magit-autorevert magit-margin magit-transient magit-process with-editor
magit-mode transient-posframe transient magit-git magit-base crm
vertico-directory cape corfu-prescient corfu-history corfu dtrt-indent
eval-sexp-fu eros lispyville lispy lispy-inline avy etags fileloop
evil-collection-edebug edebug lispy-tags mode-local zoutline elisp-def
ert ewoc evil-collection-xref xref f f-shortdoc sotlisp skeleton
elec-pair envrc inheritenv evil-surround evil-matchit-evil-setup
tabspaces dired-x vc vc-git diff-mode vc-dispatcher flyspell ispell
org-indent org-appear orgonomic org-superstar form-feed oc-basic ol-eww
eww url-queue mm-url ol-rmail ol-mhe ol-irc ol-info ol-gnus nnselect
gnus-art mm-uu mml2015 mm-view mml-smime smime gnutls dig gnus-sum shr
pixel-fill kinsoku url-file svg dom browse-url gnus-group gnus-undo
gnus-start gnus-dbus dbus xml gnus-cloud nnimap nnmail mail-source utf7
nnoo parse-time gnus-spec gnus-int gnus-range message sendmail
yank-media rfc822 mml mml-sec epa epg rfc6068 epg-config mm-decode
mm-bodies mm-encode mailabbrev gmm-utils mailheader gnus-win gnus
nnheader gnus-util mail-utils range ol-docview doc-view jka-compr
image-mode exif dired dired-loaddefs ol-bibtex bibtex iso8601 ol-bbdb
ol-w3m ol-doi org-link-doi org-download url-http url url-proxy
url-privacy url-expand url-methods url-history mailcap url-auth
mail-parse rfc2231 rfc2047 rfc2045 mm-util ietf-drums mail-prsvr
url-cookie generate-lisp-file url-domsuf url-util url-gw nsm puny async
vulpea vulpea-meta vulpea-select vulpea-buffer vulpea-db s vulpea-utils
vulpea-note org-roam-migrate org-roam-log org-roam-mode org-roam-capture
org-roam-id org-roam-node org-roam-db emacsql-sqlite-builtin sqlite
org-roam-utils org-roam-compat org-roam org-mac-link org-goto
org-capture org-attach emacsql-sqlite emacsql-sqlite-common emacsql
emacsql-compiler magit-section cursor-sensor dash evil-org-agenda
evil-org imenu ox-odt rng-loc rng-uri rng-parse rng-match rng-dt
rng-util rng-pttrn nxml-parse nxml-ns nxml-enc xmltok nxml-util ox-latex
ox-icalendar org-agenda ox-ascii ox-gfm ox-md ox-html table ox-publish
ox org-element org-persist xdg org-id org-refile avl-tree generator
org-tempo tempo ob-shell shell org ob ob-tangle ob-ref ob-lob ob-table
ob-exp org-macro org-src ob-comint org-pcomplete pcomplete org-list
org-footnote org-faces org-entities noutline outline ob-emacs-lisp
ob-core ob-eval org-cycle org-table ol org-fold org-fold-core org-keys
oc org-loaddefs cal-menu calendar cal-loaddefs org-version org-compat
org-macs format-spec undo-fu-session ws-butler yasnippet
vertico-prescient prescient char-fold vertico-mouse vertico mini-frame
better-jumper ns-auto-titlebar evil-anzu anzu which-key-posframe
posframe evil-collection-which-key which-key gcmh help-fns radix-tree
cl-print xt-mouse autorevert filenotify saveplace winner hl-line
evil-collection-ibuffer ibuffer ibuffer-loaddefs savehist delsel
popup-mode-hacks evil-collection-debug debug backtrace find-func recentf
tree-widget repeat orderless popup-mode popup-mode-settings
evil-collection annalist evil-little-word cus-edit cus-start cus-load
wid-edit pp evil evil-integration evil-maps evil-commands reveal
evil-jumps evil-command-window evil-types evil-search evil-ex
evil-macros evil-repeat evil-states evil-core advice evil-common
thingatpt rect evil-vars memoize nano-modeline nano-light-theme
face-remap nano-theme disp-table project gcmh-autoloads
copy-as-format-autoloads pdf-tools-autoloads tablist-autoloads
restclient-autoloads vterm-autoloads dumb-jump-autoloads popup-autoloads
haml-mode-autoloads emmet-mode-autoloads terraform-mode-autoloads
hcl-mode-autoloads dockerfile-mode-autoloads yaml-mode-autoloads
json-mode-autoloads json-snatcher-autoloads grip-mode-autoloads
lua-mode-autoloads bundler-autoloads inf-ruby-autoloads
ruby-refactor-autoloads evil-ruby-text-objects-autoloads
sotlisp-autoloads elisp-def-autoloads lispyville-autoloads
lispy-autoloads iedit-autoloads swiper-autoloads ivy-autoloads
zoutline-autoloads eros-autoloads eval-sexp-fu-autoloads
web-mode-autoloads ripgrep-capf-autoloads git-link-autoloads
consult-git-commit-autoloads git-timemachine-autoloads
magit-delta-autoloads xterm-color-autoloads prettier-autoloads
iter2-autoloads nvm-autoloads editorconfig-autoloads flycheck-autoloads
pkg-info-autoloads epl-autoloads lsp-ui-autoloads lsp-mode-autoloads
spinner-autoloads markdown-mode-autoloads denote-autoloads
imenu-list-autoloads org-superstar-autoloads ox-gfm-autoloads
org-pandoc-import-autoloads gnuplot-autoloads org-download-autoloads
async-autoloads org-journal-autoloads vulpea-autoloads
org-roam-autoloads emacsql-autoloads orgonomic-autoloads
org-drill-autoloads persist-autoloads org-appear-autoloads
org-mac-link-autoloads evil-org-autoloads
evil-terminal-cursor-changer-autoloads transient-posframe-autoloads
better-jumper-autoloads buffer-move-autoloads rotate-autoloads
mini-frame-autoloads embark-consult-autoloads embark-autoloads
consult-autoloads orderless-autoloads cape-autoloads
corfu-prescient-autoloads corfu-autoloads vertico-prescient-autoloads
vertico-autoloads prescient-autoloads tabspaces-autoloads
which-key-posframe-autoloads which-key-autoloads popup-mode-autoloads
hide-mode-line-autoloads evil-anzu-autoloads anzu-autoloads
titlecase-autoloads wgrep-autoloads yasnippet-autoloads
form-feed-autoloads drag-stuff-autoloads dtrt-indent-autoloads
ws-butler-autoloads evil-collection-autoloads annalist-autoloads
evil-mc-autoloads evil-numbers-autoloads speeddating-autoloads
evil-little-word-autoloads evil-matchit-autoloads
evil-nerd-commenter-autoloads evil-visualstar-autoloads
evil-surround-autoloads vundo-autoloads undo-fu-session-autoloads
ztree-autoloads dwim-shell-command-autoloads treemacs-tab-bar-autoloads
treemacs-magit-autoloads magit-autoloads git-commit-autoloads
magit-section-autoloads with-editor-autoloads treemacs-evil-autoloads
evil-autoloads goto-chg-autoloads treemacs-autoloads
ace-window-autoloads avy-autoloads pfuture-autoloads ht-autoloads
cfrs-autoloads all-the-icons-autoloads rainbow-mode-autoloads
posframe-autoloads ns-auto-titlebar-autoloads nano-modeline-autoloads
nano-theme-autoloads memoize-autoloads envrc-autoloads
inheritenv-autoloads helpful-autoloads f-autoloads elisp-refs-autoloads
s-autoloads dired-subtree-autoloads dired-hacks-utils-autoloads
dash-autoloads server pcase hydra lv url-parse auth-source eieio
eieio-core password-cache json map url-vars edmacro kmacro byte-opt
compdef derived leader-key bind-map no-littering compat
compdef-autoloads hydra-autoloads lv-autoloads leader-key-autoloads
bind-map-autoloads no-littering-autoloads compat-autoloads
elpaca-use-package use-package use-package-ensure use-package-delight
use-package-diminish use-package-bind-key bind-key easy-mmode
use-package-core elpaca-use-package-autoloads compile
text-property-search comint ansi-osc ansi-color ring time-date comp-run
cl-macs elpaca elpaca-process elpaca-autoloads comp cl-seq comp-cstr
comp-common warnings subr-x rx gv bytecomp byte-compile cl-extra
help-mode icons cl-loaddefs cl-lib display-line-numbers rmc iso-transl
tooltip cconv eldoc paren electric uniquify ediff-hook vc-hooks
lisp-float-type elisp-mode mwheel term/ns-win ns-win ucs-normalize
mule-util term/common-win tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode lisp-mode prog-mode register
page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
frame minibuffer nadvice seq simple cl-generic indonesian philippine
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp
files window text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget keymap hashtable-print-readable backquote
threads kqueue cocoa ns lcms2 multi-tty make-network-process
native-compile emacs)

Memory information:
((conses 16 1603207 1724654) (symbols 48 73841 1671) (strings 32 543300 273395)
 (string-bytes 1 15331101) (vectors 16 236430) (vector-slots 8 3553218 678973)
 (floats 8 1870 4220) (intervals 56 4454 1235) (buffers 992 19))

bug#68083: 30.0.50; Intermittent build failure with native compilation

[Eli Zaretskii]

> From: Aaron Jensen <aaronjensen@gmail.com>
> Date: Thu, 28 Dec 2023 09:05:25 -0500
> 
> 
> On macOS, I am often getting this when building Emacs from scratch. Could this be a race condition with a parallel build?
> 
> I bisected and the problematic commit is: e670412a3e101e70dc26e021f467faece8cb7f6b
> 
> In toplevel form:
> org/org-element.el:64:2: Error: Eager macro-expansion failure: (native-compiler-error (lambda (arg322 &optional arg323) (let ((f #'macroexpand)) (funcall f arg322 arg323))) "Compiling /private/var/tmp/emacs-plusA30-20231227-
> 10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln...
> File already exists: /private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln
> 
> Error: file-already-exists (\"File already exists\" \"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln\")
>   mapbacktrace(#f(compiled-function (evald func args flags) #<bytecode -0x1fcb3a734512f81>))
>   debug-early-backtrace()
>   debug-early(error (file-already-exists \"File already exists\" \"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln
> \"))
>   rename-file(\"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_08i0tl8.eln.tmp\" \"/private/var/tmp/emacs-plusA30-20231
> 227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln\")
>   comp-delete-or-replace-file(\"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln\" \"/private/var/tmp/emacs-plusA30
> -20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_08i0tl8.eln.tmp\")
>   comp--compile-ctxt-to-file(\"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln\")
>   comp-compile-ctxt-to-file(\"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln\")
>   comp-final1()
>   load-with-code-conversion(\"/private/var/tmp/emacs-int-comp-subr--trampoline-6d6163726f657870616e64_macroexpand_0-haajZW.el\" \"/private/var/tmp/emacs-int-comp-subr--trampoline-6d6163726f657870616e64_macroexpand_0-haajZW.e
> l\" nil t)
>   command-line-1((\"-l\" \"/private/var/tmp/emacs-int-comp-subr--trampoline-6d6163726f657870616e64_macroexpand_0-haajZW.el\"))
>   command-line()
>   normal-top-level()
> ")
> gmake[3]: *** [Makefile:330: org/org-element.elc] Error 1

Thanks.  Adding Andrea to the discussion.

bug#68083: 30.0.50; Intermittent build failure with native compilation

[Andrea Corallo]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Aaron Jensen <aaronjensen@gmail.com>
>> Date: Thu, 28 Dec 2023 09:05:25 -0500
>> 
>> 
>> On macOS, I am often getting this when building Emacs from scratch. Could this be a race condition with a parallel build?
>> 
>> I bisected and the problematic commit is: e670412a3e101e70dc26e021f467faece8cb7f6b
>> 
>> In toplevel form:
>> org/org-element.el:64:2: Error: Eager macro-expansion failure:
>> (native-compiler-error (lambda (arg322 &optional arg323) (let ((f
>> #'macroexpand)) (funcall f arg322 arg323))) "Compiling
>> /private/var/tmp/emacs-plusA30-20231227-
>> 10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln...
>> File already exists:
>> /private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln
>> 
>> Error: file-already-exists (\"File already exists\"
>> \"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln\")
>>   mapbacktrace(#f(compiled-function (evald func args flags) #<bytecode -0x1fcb3a734512f81>))
>>   debug-early-backtrace()
>>   debug-early(error (file-already-exists \"File already exists\"
>> \"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln
>> \"))
>>   rename-file(\"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_08i0tl8.eln.tmp\"
>> \"/private/var/tmp/emacs-plusA30-20231
>> 227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln\")
>>   comp-delete-or-replace-file(\"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln\"
>> \"/private/var/tmp/emacs-plusA30
>> -20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_08i0tl8.eln.tmp\")
>>   comp--compile-ctxt-to-file(\"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln\")
>>   comp-compile-ctxt-to-file(\"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln\")
>>   comp-final1()
>>   load-with-code-conversion(\"/private/var/tmp/emacs-int-comp-subr--trampoline-6d6163726f657870616e64_macroexpand_0-haajZW.el\"
>> \"/private/var/tmp/emacs-int-comp-subr--trampoline-6d6163726f657870616e64_macroexpand_0-haajZW.e
>> l\" nil t)
>>   command-line-1((\"-l\" \"/private/var/tmp/emacs-int-comp-subr--trampoline-6d6163726f657870616e64_macroexpand_0-haajZW.el\"))
>>   command-line()
>>   normal-top-level()
>> ")
>> gmake[3]: *** [Makefile:330: org/org-element.elc] Error 1
>
> Thanks.  Adding Andrea to the discussion.

Hi all,

reading from emacs-devel... is it still confirmed that the commit that
introduced this is e670412a3e101e70dc26e021f467faece8cb7f6b?

Thanks

  Andrea

bug#68083: 30.0.50; Intermittent build failure with native compilation

[Aaron Jensen]

[-- Attachment #1: Type: text/plain, Size: 3565 bytes --]

Yes, that's what I've found. I can also confirm that compiling with 1
thread works around it. It's only a problem w/ gmake -jN where N is > 1 (I
run w/ 8 or 12 or so typically).


Aaron


On Fri, Dec 29, 2023 at 1:43 PM, Andrea Corallo <acorallo@gnu.org> wrote:

> Eli Zaretskii <eliz@gnu.org> writes:
>
> From: Aaron Jensen <aaronjensen@gmail.com>
> Date: Thu, 28 Dec 2023 09:05:25 -0500
>
> On macOS, I am often getting this when building Emacs from scratch. Could
> this be a race condition with a parallel build?
>
> I bisected and the problematic commit is:
> e670412a3e101e70dc26e021f467faece8cb7f6b
>
> In toplevel form:
> org/org-element.el:64:2: Error: Eager macro-expansion failure:
> (native-compiler-error (lambda (arg322 &optional arg323) (let ((f
> #'macroexpand)) (funcall f arg322 arg323))) "Compiling
> /private/var/tmp/emacs-plusA30-20231227-
> 10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln...
> File already exists:
> /private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln
>
>
> Error: file-already-exists (\"File already exists\"
> \"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln\")
> mapbacktrace(#f(compiled-function (evald func args flags) #<bytecode
> -0x1fcb3a734512f81>)) debug-early-backtrace()
> debug-early(error (file-already-exists \"File already exists\"
> \"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln
>
> \"))
> rename-file(\"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_08i0tl8.eln.tmp\"
>
> \"/private/var/tmp/emacs-plusA30-20231
> 227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln\")
> comp-delete-or-replace-file(\"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln\"
>
> \"/private/var/tmp/emacs-plusA30
> -20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_08i0tl8.eln.tmp\")
> comp--compile-ctxt-to-file(\"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln\")
> comp-compile-ctxt-to-file(\"/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln\")
> comp-final1()
> load-with-code-conversion(\"/private/var/tmp/emacs-int-comp-subr--trampoline-6d6163726f657870616e64_macroexpand_0-haajZW.el\"
>
> \"/private/var/tmp/emacs-int-comp-subr--trampoline-6d6163726f657870616e64_macroexpand_0-haajZW.e
> l\" nil t)
> command-line-1((\"-l\"
> \"/private/var/tmp/emacs-int-comp-subr--trampoline-6d6163726f657870616e64_macroexpand_0-haajZW.el\"))
> command-line()
> normal-top-level()
> ")
> gmake[3]: *** [Makefile:330: org/org-element.elc] Error 1
>
> Thanks. Adding Andrea to the discussion.
>
> Hi all,
>
> reading from emacs-devel... is it still confirmed that the commit that
> introduced this is e670412a3e101e70dc26e021f467faece8cb7f6b?
>
> Thanks
>
> Andrea
>

[-- Attachment #2: Type: text/html, Size: 5168 bytes --]

bug#68083: 30.0.50; Intermittent build failure with native compilation

[Andrea Corallo]

Aaron Jensen <aaronjensen@gmail.com> writes:

> Yes, that's what I've found. I can also confirm that compiling with 1 thread works around it. It's only a problem w/
> gmake -jN where N is > 1 (I run w/ 8 or 12 or so typically).
>
> Aaron

Intresting, adding Jens, hopefully he has some good idea.

Maybe you could re-add 'macroexpand' and 'rename-buffer' to
'native-comp-never-optimize-functions' and discover which one of the two
is triggering the bug?

Thanks

  Andrea

bug#68083: 30.0.50; Intermittent build failure with native compilation

[Aaron Jensen]

[-- Attachment #1: Type: text/plain, Size: 818 bytes --]

On Fri, Dec 29, 2023 at 3:17 PM, Andrea Corallo <acorallo@gnu.org> wrote:

> Aaron Jensen <aaronjensen@gmail.com> writes:
>
> Yes, that's what I've found. I can also confirm that compiling with 1
> thread works around it. It's only a problem w/ gmake -jN where N is > 1 (I
> run w/ 8 or 12 or so typically).
>
> Aaron
>
> Intresting, adding Jens, hopefully he has some good idea.
>
> Maybe you could re-add 'macroexpand' and 'rename-buffer' to
> 'native-comp-never-optimize-functions' and discover which one of the two
> is triggering the bug?
>


I can't try it just now, but my trace includes: `File already exists:
/private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln`
so I'm guessing it's macroexpand.

Aaron

[-- Attachment #2: Type: text/html, Size: 2194 bytes --]

bug#68083: 30.0.50; Intermittent build failure with native compilation

[Aaron Jensen]

[-- Attachment #1: Type: text/plain, Size: 1262 bytes --]

Just to confirm, adding macroexpand to native-comp-never-optimize-functions
allows me to build successfully.

It also looks like comp-delete-or-replace-file can be updated to protect
rename-file against file-already-exists like it does for Windows. That
would also likely solve the problem if you want to be able to optimize
macroexpand.




Aaron


On Fri, Dec 29, 2023 at 3:26 PM, Aaron Jensen <aaronjensen@gmail.com> wrote:

> On Fri, Dec 29, 2023 at 3:17 PM, Andrea Corallo <acorallo@gnu.org> wrote:
>
> Aaron Jensen <aaronjensen@gmail.com> writes:
>
> Yes, that's what I've found. I can also confirm that compiling with 1
> thread works around it. It's only a problem w/ gmake -jN where N is > 1 (I
> run w/ 8 or 12 or so typically).
>
> Aaron
>
> Intresting, adding Jens, hopefully he has some good idea.
>
> Maybe you could re-add 'macroexpand' and 'rename-buffer' to
> 'native-comp-never-optimize-functions' and discover which one of the two
> is triggering the bug?
>
>
>
> I can't try it just now, but my trace includes: `File already exists:
> /private/var/tmp/emacs-plusA30-20231227-10652-1cz0rs/.brew_home/.emacs.d/eln-cache/30.0.50-69afc345/subr--trampoline-6d6163726f657870616e64_macroexpand_0.eln`
> so I'm guessing it's macroexpand.
>
> Aaron
>

[-- Attachment #2: Type: text/html, Size: 4046 bytes --]

bug#68083: 30.0.50; Intermittent build failure with native compilation

[Eli Zaretskii]

> From: Aaron Jensen <aaronjensen@gmail.com>
> Date: Fri, 29 Dec 2023 16:07:38 -0500
> Cc: Eli Zaretskii <eliz@gnu.org>, 68083@debbugs.gnu.org, 
> 	Jens Schmidt <jschmidt4gnu@vodafonemail.de>
> 
> Just to confirm, adding macroexpand to native-comp-never-optimize-functions allows me to build
> successfully.
> 
> It also looks like comp-delete-or-replace-file can be updated to protect rename-file against
> file-already-exists like it does for Windows. That would also likely solve the problem if you want to be
> able to optimize macroexpand.

Are you sure?  We do that on Windows because Windows doesn't allow us
to delete a file that is open by another program.  That shouldn't
happen on Posix systems, so I think what you see here is due to a race
between checking whether a file exists and renaming it, which is a
different problem.

However, feel free to try the same trick we use on Windows and see
whether it helps.

bug#68083: 30.0.50; Intermittent build failure with native compilation

[Aaron Jensen]

On Sat, Dec 30, 2023 at 1:47 AM Eli Zaretskii <eliz@gnu.org> wrote:
>
> > From: Aaron Jensen <aaronjensen@gmail.com>
> > Date: Fri, 29 Dec 2023 16:07:38 -0500
> > Cc: Eli Zaretskii <eliz@gnu.org>, 68083@debbugs.gnu.org,
> >       Jens Schmidt <jschmidt4gnu@vodafonemail.de>
> >
> > Just to confirm, adding macroexpand to native-comp-never-optimize-functions allows me to build
> > successfully.
> >
> > It also looks like comp-delete-or-replace-file can be updated to protect rename-file against
> > file-already-exists like it does for Windows. That would also likely solve the problem if you want to be
> > able to optimize macroexpand.
>
> Are you sure?  We do that on Windows because Windows doesn't allow us
> to delete a file that is open by another program.  That shouldn't
> happen on Posix systems, so I think what you see here is due to a race
> between checking whether a file exists and renaming it, which is a
> different problem.
>
> However, feel free to try the same trick we use on Windows and see
> whether it helps.

This fixes it for me:

diff --git a/lisp/emacs-lisp/comp.el b/lisp/emacs-lisp/comp.el
index 3b2fd25e61c..80088f935a4 100644
--- a/lisp/emacs-lisp/comp.el
+++ b/lisp/emacs-lisp/comp.el
@@ -3341,7 +3341,11 @@ comp-delete-or-replace-file
         ;; is currently loaded.
         (t (delete-file oldfile)
            (when newfile
-             (rename-file newfile oldfile)))))
+             (condition-case _
+                 (rename-file newfile oldfile)
+               (file-already-exists
+                (delete-file newfile)
+                t))))))

 (defun comp--native-compile (function-or-file &optional with-late-load output)
   "Compile FUNCTION-OR-FILE into native code.

I imagine that this is worth doing just to make this operation
parallel-safe, but I wonder why macroexpand is the only instance of
this happening. I don't know if macroexpand should still be in
native-comp-never-optimize-functions or not (i.e., is there another
reason it was there other than to avoid this crash?)

Aaron

bug#68083: 30.0.50; Intermittent build failure with native compilation

[Eli Zaretskii]

> From: Aaron Jensen <aaronjensen@gmail.com>
> Date: Sat, 30 Dec 2023 10:29:16 -0500
> Cc: acorallo@gnu.org, 68083@debbugs.gnu.org, jschmidt4gnu@vodafonemail.de
> 
> On Sat, Dec 30, 2023 at 1:47 AM Eli Zaretskii <eliz@gnu.org> wrote:
> >
> > Are you sure?  We do that on Windows because Windows doesn't allow us
> > to delete a file that is open by another program.  That shouldn't
> > happen on Posix systems, so I think what you see here is due to a race
> > between checking whether a file exists and renaming it, which is a
> > different problem.
> >
> > However, feel free to try the same trick we use on Windows and see
> > whether it helps.
> 
> This fixes it for me:
> 
> diff --git a/lisp/emacs-lisp/comp.el b/lisp/emacs-lisp/comp.el
> index 3b2fd25e61c..80088f935a4 100644
> --- a/lisp/emacs-lisp/comp.el
> +++ b/lisp/emacs-lisp/comp.el
> @@ -3341,7 +3341,11 @@ comp-delete-or-replace-file
>          ;; is currently loaded.
>          (t (delete-file oldfile)
>             (when newfile
> -             (rename-file newfile oldfile)))))
> +             (condition-case _
> +                 (rename-file newfile oldfile)
> +               (file-already-exists
> +                (delete-file newfile)
> +                t))))))

What happens if, instead of wrapping rename-file in condition-case,
you change that to say

  (t (if newfile
	 (rename-file newfile oldfile)
       (delete-file oldfile))

> I imagine that this is worth doing just to make this operation
> parallel-safe, but I wonder why macroexpand is the only instance of
> this happening. I don't know if macroexpand should still be in
> native-comp-never-optimize-functions or not (i.e., is there another
> reason it was there other than to avoid this crash?)

rename-file is supposed to be an atomic operation on Posix
filesystems, so I don't quite understand why you see what you see, and
I'm hesitant to sweep under the carpet a problem we don't understand.

bug#68083: 30.0.50; Intermittent build failure with native compilation

[Aaron Jensen]

On Sat, Dec 30, 2023 at 12:43 PM Eli Zaretskii <eliz@gnu.org> wrote:
>
> > From: Aaron Jensen <aaronjensen@gmail.com>
> > Date: Sat, 30 Dec 2023 10:29:16 -0500
> > Cc: acorallo@gnu.org, 68083@debbugs.gnu.org, jschmidt4gnu@vodafonemail.de
> >
> > On Sat, Dec 30, 2023 at 1:47 AM Eli Zaretskii <eliz@gnu.org> wrote:
> > >
> > > Are you sure?  We do that on Windows because Windows doesn't allow us
> > > to delete a file that is open by another program.  That shouldn't
> > > happen on Posix systems, so I think what you see here is due to a race
> > > between checking whether a file exists and renaming it, which is a
> > > different problem.
> > >
> > > However, feel free to try the same trick we use on Windows and see
> > > whether it helps.
> >
> > This fixes it for me:
> >
> > diff --git a/lisp/emacs-lisp/comp.el b/lisp/emacs-lisp/comp.el
> > index 3b2fd25e61c..80088f935a4 100644
> > --- a/lisp/emacs-lisp/comp.el
> > +++ b/lisp/emacs-lisp/comp.el
> > @@ -3341,7 +3341,11 @@ comp-delete-or-replace-file
> >          ;; is currently loaded.
> >          (t (delete-file oldfile)
> >             (when newfile
> > -             (rename-file newfile oldfile)))))
> > +             (condition-case _
> > +                 (rename-file newfile oldfile)
> > +               (file-already-exists
> > +                (delete-file newfile)
> > +                t))))))
>
> What happens if, instead of wrapping rename-file in condition-case,
> you change that to say
>
>   (t (if newfile
>          (rename-file newfile oldfile)
>        (delete-file oldfile))

I don't really understand this change. The previous version of the
code wraps the rename in a (when newfile) and deletes the oldfile
right before that. The deletion would always be necessary if the
oldfile exists unless OK-IF-ALREADY-EXISTS is specified in
rename-file.

I tried it anyway for diligence, and I get the exact same behavior
(the rename fails because the file exists).

> > I imagine that this is worth doing just to make this operation
> > parallel-safe, but I wonder why macroexpand is the only instance of
> > this happening. I don't know if macroexpand should still be in
> > native-comp-never-optimize-functions or not (i.e., is there another
> > reason it was there other than to avoid this crash?)
>
> rename-file is supposed to be an atomic operation on Posix
> filesystems, so I don't quite understand why you see what you see, and
> I'm hesitant to sweep under the carpet a problem we don't understand.

Indeed, but we aren't specifying OK-IF-ALREADY-EXISTS, so if another
process does the same compile at the same time (and therefore the same
rename), they will conflict. We could specify OK-IF-ALREADY-EXISTS
instead and that works as well:

diff --git a/lisp/emacs-lisp/comp.el b/lisp/emacs-lisp/comp.el
index 3b2fd25e61c..d56c69d9470 100644
--- a/lisp/emacs-lisp/comp.el
+++ b/lisp/emacs-lisp/comp.el
@@ -3341,7 +3341,7 @@ comp-delete-or-replace-file
         ;; is currently loaded.
         (t (delete-file oldfile)
            (when newfile
-             (rename-file newfile oldfile)))))
+             (rename-file newfile oldfile t)))))

 (defun comp--native-compile (function-or-file &optional with-late-load output)
   "Compile FUNCTION-OR-FILE into native code.

Aaron

bug#68083: 30.0.50; Intermittent build failure with native compilation

[Eli Zaretskii]

> From: Aaron Jensen <aaronjensen@gmail.com>
> Date: Sat, 30 Dec 2023 13:06:55 -0500
> Cc: acorallo@gnu.org, 68083@debbugs.gnu.org, jschmidt4gnu@vodafonemail.de
> 
> On Sat, Dec 30, 2023 at 12:43 PM Eli Zaretskii <eliz@gnu.org> wrote:
> >
> > What happens if, instead of wrapping rename-file in condition-case,
> > you change that to say
> >
> >   (t (if newfile
> >          (rename-file newfile oldfile)
> >        (delete-file oldfile))
> 
> I don't really understand this change. The previous version of the
> code wraps the rename in a (when newfile) and deletes the oldfile
> right before that. The deletion would always be necessary if the
> oldfile exists unless OK-IF-ALREADY-EXISTS is specified in
> rename-file.

Sorry, I meant to add the OK-IF-ALREADY-EXISTS argument non-nil, of
course.

The point is that you can rename a file if the OLDFILE exists with no
problem, and that is supposed to be an atomic operation, so no race
conditions.

> Indeed, but we aren't specifying OK-IF-ALREADY-EXISTS, so if another
> process does the same compile at the same time (and therefore the same
> rename), they will conflict. We could specify OK-IF-ALREADY-EXISTS
> instead and that works as well:

Then I prefer this version.

bug#68083: 30.0.50; Intermittent build failure with native compilation

[Aaron Jensen]

[-- Attachment #1: Type: text/plain, Size: 1561 bytes --]

 Works for me, though is it still an outstanding question as to whether or
not we should add macro expand back?

Also yeah if we add the t we can do what you suggested where it’s either a
delete or rename force.


Aaron

On Sat, Dec 30 2023 at 1:46 PM, Eli Zaretskii <eliz@gnu.org> wrote:

> From: Aaron Jensen <aaronjensen@gmail.com>
> Date: Sat, 30 Dec 2023 13:06:55 -0500
> Cc: acorallo@gnu.org, 68083@debbugs.gnu.org, jschmidt4gnu@vodafonemail.de
>
> On Sat, Dec 30, 2023 at 12:43 PM Eli Zaretskii <eliz@gnu.org> wrote:
> >
> > What happens if, instead of wrapping rename-file in condition-case,
> > you change that to say
> >
> > (t (if newfile
> > (rename-file newfile oldfile)
> > (delete-file oldfile))
>
> I don't really understand this change. The previous version of the code
> wraps the rename in a (when newfile) and deletes the oldfile right before
> that. The deletion would always be necessary if the oldfile exists unless
> OK-IF-ALREADY-EXISTS is specified in rename-file.
>
> Sorry, I meant to add the OK-IF-ALREADY-EXISTS argument non-nil, of
> course.
>
> The point is that you can rename a file if the OLDFILE exists with no
> problem, and that is supposed to be an atomic operation, so no race
> conditions.
>
> Indeed, but we aren't specifying OK-IF-ALREADY-EXISTS, so if another
> process does the same compile at the same time (and therefore the same
> rename), they will conflict. We could specify OK-IF-ALREADY-EXISTS instead
> and that works as well:
>
> Then I prefer this version.
>

[-- Attachment #2: Type: text/html, Size: 2522 bytes --]

bug#68083: 30.0.50; Intermittent build failure with native compilation

[Aaron Jensen]

[-- Attachment #1.1: Type: text/plain, Size: 1742 bytes --]

Here's a patch, if helpful. I've tested it and can build with it.


Aaron


On Sat, Dec 30, 2023 at 3:54 PM, Aaron Jensen <aaronjensen@gmail.com> wrote:

> Works for me, though is it still an outstanding question as to whether or
> not we should add macro expand back?
>
> Also yeah if we add the t we can do what you suggested where it’s either a
> delete or rename force.
>
>
> Aaron
>
> On Sat, Dec 30 2023 at 1:46 PM, Eli Zaretskii <eliz@gnu.org> wrote:
>
> From: Aaron Jensen <aaronjensen@gmail.com>
> Date: Sat, 30 Dec 2023 13:06:55 -0500
> Cc: acorallo@gnu.org, 68083@debbugs.gnu.org, jschmidt4gnu@vodafonemail.de
>
> On Sat, Dec 30, 2023 at 12:43 PM Eli Zaretskii <eliz@gnu.org> wrote:
> >
> > What happens if, instead of wrapping rename-file in condition-case,
> > you change that to say
> >
> > (t (if newfile
> > (rename-file newfile oldfile)
> > (delete-file oldfile))
>
> I don't really understand this change. The previous version of the code
> wraps the rename in a (when newfile) and deletes the oldfile right before
> that. The deletion would always be necessary if the oldfile exists unless
> OK-IF-ALREADY-EXISTS is specified in rename-file.
>
> Sorry, I meant to add the OK-IF-ALREADY-EXISTS argument non-nil, of
> course.
>
> The point is that you can rename a file if the OLDFILE exists with no
> problem, and that is supposed to be an atomic operation, so no race
> conditions.
>
> Indeed, but we aren't specifying OK-IF-ALREADY-EXISTS, so if another
> process does the same compile at the same time (and therefore the same
> rename), they will conflict. We could specify OK-IF-ALREADY-EXISTS instead
> and that works as well:
>
> Then I prefer this version.
>
>

[-- Attachment #1.2: Type: text/html, Size: 4328 bytes --]

[-- Attachment #2: 0001-comp.el-comp-delete-or-replace-file-Fix-parallel-com.patch --]
[-- Type: application/octet-stream, Size: 1073 bytes --]

From c1ab27179e68b94c9adb4905c677c7f344c09e6d Mon Sep 17 00:00:00 2001
From: Aaron Jensen <aaronjensen@gmail.com>
Date: Sat, 30 Dec 2023 18:06:45 -0500
Subject: [PATCH] * comp.el (comp-delete-or-replace-file): Fix parallel
 compilation

---
 lisp/emacs-lisp/comp.el | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/lisp/emacs-lisp/comp.el b/lisp/emacs-lisp/comp.el
index 3b2fd25e61c..f9eeef1b9e8 100644
--- a/lisp/emacs-lisp/comp.el
+++ b/lisp/emacs-lisp/comp.el
@@ -3339,9 +3339,9 @@ comp-delete-or-replace-file
         ;; Remove the old eln instead of copying the new one into it
         ;; to get a new inode and prevent crashes in case the old one
         ;; is currently loaded.
-        (t (delete-file oldfile)
-           (when newfile
-             (rename-file newfile oldfile)))))
+        (t (if newfile
+               (rename-file newfile oldfile t)
+             (delete-file oldfile)))))
 
 (defun comp--native-compile (function-or-file &optional with-late-load output)
   "Compile FUNCTION-OR-FILE into native code.
-- 
2.42.1

bug#68083: 30.0.50; Intermittent build failure with native compilation

[Eli Zaretskii]

> From: Aaron Jensen <aaronjensen@gmail.com>
> Date: Sat, 30 Dec 2023 14:54:53 -0600
> Cc: Andrea Corallo <acorallo@gnu.org>, 68083@debbugs.gnu.org, 
> 	Jens Schmidt <jschmidt4gnu@vodafonemail.de>
> 
> Works for me, though is it still an outstanding question as to whether or not we should add macro
> expand back?

Andrea, WDYT?

bug#68083: 30.0.50; Intermittent build failure with native compilation

[Jens Schmidt via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On 2023-12-29  21:17, Andrea Corallo wrote:

> Intresting, adding Jens, hopefully he has some good idea.

Thanks for inviting me to the party and sorry for attending late,
but I'm grounded by a flu ... will check later if there is still
anything to check.

bug#68083: 30.0.50; Intermittent build failure with native compilation

[Andrea Corallo]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Aaron Jensen <aaronjensen@gmail.com>
>> Date: Sat, 30 Dec 2023 14:54:53 -0600
>> Cc: Andrea Corallo <acorallo@gnu.org>, 68083@debbugs.gnu.org, 
>> 	Jens Schmidt <jschmidt4gnu@vodafonemail.de>
>> 
>> Works for me, though is it still an outstanding question as to whether or not we should add macro
>> expand back?
>
> Andrea, WDYT?

I think we should not re-add it unless there's a specific reason.

If the proposed patch solves the issue I guess we should be fine with
that.

Thanks

  Andrea

bug#68083: 30.0.50; Intermittent build failure with native compilation

[Eli Zaretskii]

> From: Andrea Corallo <acorallo@gnu.org>
> Cc: Aaron Jensen <aaronjensen@gmail.com>,  68083@debbugs.gnu.org,
>   jschmidt4gnu@vodafonemail.de
> Date: Mon, 01 Jan 2024 14:39:38 -0500
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: Aaron Jensen <aaronjensen@gmail.com>
> >> Date: Sat, 30 Dec 2023 14:54:53 -0600
> >> Cc: Andrea Corallo <acorallo@gnu.org>, 68083@debbugs.gnu.org, 
> >> 	Jens Schmidt <jschmidt4gnu@vodafonemail.de>
> >> 
> >> Works for me, though is it still an outstanding question as to whether or not we should add macro
> >> expand back?
> >
> > Andrea, WDYT?
> 
> I think we should not re-add it unless there's a specific reason.
> 
> If the proposed patch solves the issue I guess we should be fine with
> that.

Thanks, so I installed the last patch, and I'm closing this bug.