bug#46972: 28.0.50; [feature/native-comp] Emacs locks itself during native compilation because of permission denied

bug#46972: 28.0.50; [feature/native-comp] Emacs locks itself during native compilation because of permission denied

[Matt M]

[-- Attachment #1: Type: text/plain, Size: 10873 bytes --]

I'm on windows using latest native compilation. I noticed that when doing
doom sync (I use Doom emacs) it would hang on the native compilation
step. I looked at the opened emacs processes in the task manager
during this bug to try to find a file with which I could make the bug
trigger again. In the task manager I see that about 10 emacs processes
are stuck on 10 files.

Using this file: https://pastebin.com/z4wLheXa as
emacs-async-comp-ox-ascii.el, I call the following command:

emacs -Q --batch -l c:/Users/Matt/emacs-async-comp-ox-ascii.el
> Compiling c:/Users/Matt/.emacs.d/.local/straight/build-28.0.50/org-mode/ox-ascii.el...

It never finishes. If I interrupt the process I get the following
backtrace:

Debugger entered--Lisp error: (file-error "Removing old name" "Permission denied" "c:/Users/Matt/.emacs.d/.local/cache/eln/28.0.50-4c...")
  kill-emacs(t)
  command-line()
  normal-top-level()

Removing old name: Permission denied, c:/Users/Matt/.emacs.d/.local/cache/eln/28.0.50-4c6fa13f/ox-ascii-dcca1ba0-825ea35cp6iXUJ.eln.old
Debugger entered--Lisp error: (file-error "Removing old name" "Permission denied" "c:/Users/Matt/.emacs.d/.local/cache/eln/28.0.50-4c...")
  command-error-default-function((file-error "Removing old name" "Permission denied" "c:/Users/Matt/.emacs.d/.local/cache/eln/28.0.50-4c...") "" debug)
  apply(command-error-default-function ((file-error "Removing old name" "Permission denied" "c:/Users/Matt/.emacs.d/.local/cache/eln/28.0.50-4c...") "" debug))
  #f(advice-wrapper :after command-error-default-function help-command-error-confusable-suggestions)((file-error "Removing old name" "Permission denied" "c:/Users/Matt/.emacs.d/.local/cache/eln/28.0.50-4c...") "" debug)

It renamed a file from file.eln to file.eln.old, then tries to delete
the file.eln.old but gets a Permission denied error.

If I start the same command again without interrupting, and try to
delete the file.eln.old manually, I get the error:
Can't remove file because the file is opened in emacs.exe

And the only emacs.exe process runnin on the computer is the one doing
the native compilation, the one which created the file.eln.old.

So that seems to indicate that the emacs process that renames the file
doesn't have the permission to remove the same file because of it being
opened in itself.



In GNU Emacs 28.0.50 (build 1, x86_64-w64-mingw32)
of 2021-03-05 built on DESKTOP-MATT
Repository revision: 552ef6d6c0733b864bcb14eeb6183d7e64df3b80
Repository branch: feature/native-comp
Windowing system distributor 'Microsoft Corp.', version 10.0.19042
System Description: Microsoft Windows 10 Education (v10.0.2009.19042.804)

Configured using:
'configure --with-native-compilation --with-gnutls --with-jpeg
--with-png --with-rsvg --with-xml2 --without-imagemagick --without-pop
--without-dbus'

Configured features:
ACL GIF GMP GNUTLS HARFBUZZ JPEG JSON LIBXML2 MODULES NATIVE_COMP NOTIFY
W32NOTIFY PDUMPER PNG RSVG SOUND THREADS TIFF TOOLKIT_SCROLL_BARS XPM
ZLIB

Important settings:
  value of $LC_CTYPE: fr_FR.UTF-8
  value of $LANG: FRA
  locale-coding-system: utf-8

Major mode: DOOM v2.0.9

Minor modes in effect:
  gcmh-mode: t
  global-hl-line-mode: t
  hl-line-mode: t
  winner-mode: t
  show-paren-mode: t
  ws-butler-global-mode: t
  global-undo-fu-session-mode: t
  undo-fu-session-mode: t
  undo-fu-mode: t
  global-flycheck-mode: t
  smartparens-global-mode: t
  which-key-mode: t
  better-jumper-mode: t
  better-jumper-local-mode: t
  company-box-mode: t
  global-company-mode: t
  company-mode: t
  ivy-prescient-mode: t
  prescient-persist-mode: t
  ivy-rich-project-root-cache-mode: t
  ivy-rich-mode: t
  ivy-mode: t
  evil-goggles-mode: t
  evil-escape-mode: t
  evil-snipe-override-mode: t
  evil-snipe-mode: t
  evil-snipe-override-local-mode: t
  evil-snipe-local-mode: t
  savehist-mode: t
  solaire-global-mode: t
  doom-modeline-mode: t
  key-chord-mode: t
  evil-repeat-motion-mode: t
  global-git-commit-mode: t
  org-roam-mode: t
  persp-mode: t
  shell-dirtrack-mode: t
  evil-mode: t
  evil-local-mode: t
  +popup-mode: t
  general-override-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  window-divider-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  size-indication-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
c:/Users/Matt/.emacs.d/.local/straight/build-28.0.50/eri/eri hides c:/Users/Matt/.emacs.d/.local/straight/build-28.0.50/agda2-mode/eri
c:/Users/Matt/.emacs.d/.local/straight/build-28.0.50/annotation/annotation hides c:/Users/Matt/.emacs.d/.local/straight/build-28.0.50/agda2-mode/annotation
c:/Users/Matt/.emacs.d/.local/straight/build-28.0.50/agda2-mode/agda-input hides c:/Users/Matt/.emacs.d/.local/straight/build-28.0.50/agda-input/agda-input

Features:
(shadow sort whitespace vi-tilde-fringe display-line-numbers
adaptive-wrap gcmh hl-line winner paren ws-butler undo-fu-session
undo-fu flycheck-popup-tip evil-collection-popup popup
evil-collection-flycheck flycheck nav-flash hide-mode-line mail-extr
smartparens-config smartparens-text smartparens emacsbug sendmail
char-fold cursor-sensor amx evil-collection-which-key which-key
better-jumper company-box company-box-doc frame-local company-box-icons
dash-functional company-capf company ivy-prescient prescient
evil-collection-ivy ivy-avy avy all-the-icons-ivy ivy-rich counsel xdg
ivy-xref evil-collection-xref xref project swiper ivy delsel ivy-faces
ivy-overlay colir color evil-goggles pulse evil-easymotion evil-escape
evil-snipe org-agenda lv doom-snippets doom-snippets-lib yasnippet
evil-collection-elisp-mode elisp-mode savehist recentf tree-widget
face-remap persistent-soft list-utils pcache eieio-base font-utils
unicode-fonts doom-themes-ext-org doom-themes-ext-neotree
doom-themes-ext-treemacs solaire-mode doom-one-theme doom-themes
doom-themes-base dtrt-indent doom-modeline doom-modeline-segments
doom-modeline-env doom-modeline-core shrink-path all-the-icons
all-the-icons-faces data-material data-weathericons data-octicons
data-fileicons data-faicons data-alltheicons key-chord
evil-repeat-motion deft orgit evil-collection-magit-todos magit-todos
pcre2el rxt re-builder hl-todo async evil-collection-grep grep
evil-collection-compile compile github-review ghub-graphql treepy gsexp
ghub url-http url-gw nsm url-auth gnutls deferred a
evil-collection-magit magit-autoloads magit-submodule magit-obsolete
magit-popup magit-blame magit-stash magit-reflog magit-bisect magit-push
magit-pull magit-fetch magit-clone magit-remote magit-commit
magit-sequence magit-notes magit-worktree magit-tag magit-merge
magit-branch magit-reset magit-files magit-refs magit-status magit
core-packages package browse-url url url-proxy url-privacy url-expand
url-methods url-history url-cookie url-domsuf url-util mailcap
url-handlers magit-repos magit-apply magit-wip magit-log magit-diff
smerge-mode diff git-commit evil-collection-log-edit log-edit message
rmc puny rfc822 mml mml-sec evil-collection-epa epa epg epg-config
gnus-util rmail rmail-loaddefs text-property-search mm-decode mm-bodies
mm-encode mail-parse rfc2231 rfc2047 rfc2045 mm-util ietf-drums
mail-prsvr mailabbrev mail-utils gmm-utils mailheader pcvs-util add-log
magit-core magit-autorevert autorevert filenotify magit-margin
magit-transient magit-process with-editor evil-collection-term term
ehelp esh-help evil-collection-man man em-unix eshell-z em-dirs esh-var
evil-collection-eshell em-prompt eshell-did-you-mean esh-mode eshell
esh-cmd esh-ext esh-opt esh-proc esh-io esh-arg esh-module esh-groups
esh-util evil-collection-dired dired dired-loaddefs server magit-mode
transient help-mode magit-git magit-section benchmark magit-utils
which-func evil-collection-imenu imenu evil-collection-vc-git vc-git
evil-collection-diff-mode diff-mode ido crm org-roam org-roam-link
org-roam-graph xml org-roam-doctor org-roam-dailies org-roam-capture
org-roam-db emacsql-sqlite3 emacsql emacsql-compiler org-capture
org-roam-completion org-roam-buffer org-roam-faces org-roam-macs
org-roam-compat f s dash org-id org-refile smartparens-org org-yt
org-element avl-tree generator org ob ob-tangle ob-ref ob-lob ob-table
ob-exp org-macro org-footnote org-src ob-comint org-pcomplete org-list
org-faces org-entities time-date noutline outline org-version
ob-emacs-lisp ob-core ob-eval org-table ol org-keys org-compat org-macs
org-loaddefs format-spec find-func evil-collection-calendar
evil-collection-custom cus-edit cus-start cus-load wid-edit
evil-collection-comint evil-collection annalist cal-menu calendar
cal-loaddefs persp-mode let-alist ibuf-macs evil evil-integration
evil-maps evil-commands ffap url-parse auth-source eieio eieio-core
eieio-loaddefs password-cache json map url-vars reveal flyspell ispell
evil-jumps evil-command-window evil-types evil-search shell pcomplete
comint ansi-color evil-macros evil-repeat evil-states evil-core advice
evil-common windmove thingatpt rect evil-digraphs evil-vars ring derived
core-editor core-projects core-ui edmacro kmacro easy-mmode comp
comp-cstr warnings rx core-keybinds pp general cl-extra easymenu seq
byte-opt cl-seq use-package-core bytecomp byte-compile cconv
core-modules realgud-recursive-autoloads cl core core-lib cl-macs gv
cl-loaddefs cl-lib subr-x iso-transl tooltip eldoc electric uniquify
ediff-hook vc-hooks lisp-float-type mwheel dos-w32 ls-lisp disp-table
term/w32-win w32-win w32-vars term/common-win tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch timer
select scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame minibuffer cl-generic cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese composite charscript charprop case-table epa-hook
jka-cmpr-hook simple abbrev obarray cl-preloaded nadvice button loaddefs
faces cus-face pcase macroexp files window text-properties overlay sha1
md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote threads w32notify w32 multi-tty
make-network-process nativecomp emacs)

Memory information:
((conses 16 1092168 1889617)
(symbols 48 49260 2178)
(strings 32 164215 99347)
(string-bytes 1 5915413)
(vectors 16 64852)
(vector-slots 8 1222623 822052)
(floats 8 1154 2009)
(intervals 56 811 188)
(buffers 992 13))

[-- Attachment #2: Type: text/html, Size: 18270 bytes --]

bug#46972: 28.0.50; [feature/native-comp] Emacs locks itself during native compilation because of permission denied

[Eli Zaretskii]

> From: Matt M <mmerino@outlook.fr>
> Date: Sat, 6 Mar 2021 17:23:34 +0000
> 
> I'm on windows using latest native compilation. I noticed that when doing
> doom sync (I use Doom emacs) it would hang on the native compilation
> step. I looked at the opened emacs processes in the task manager
> during this bug to try to find a file with which I could make the bug
> trigger again. In the task manager I see that about 10 emacs processes
> are stuck on 10 files.
> 
> Using this file: https://pastebin.com/z4wLheXa as
> emacs-async-comp-ox-ascii.el, I call the following command:
> 
> emacs -Q --batch -l c:/Users/Matt/emacs-async-comp-ox-ascii.el
> > Compiling c:/Users/Matt/.emacs.d/.local/straight/build-28.0.50/org-mode/ox-ascii.el...
> 
> It never finishes.

Does it really "never" finish, or does it just take a very long time?
How long did you wait for it to finish?

bug#46972: RE : bug#46972: 28.0.50; [feature/native-comp] Emacs locks itself during native compilation because of permission denied

[Matt M]

[-- Attachment #1: Type: text/plain, Size: 1782 bytes --]

It doesn’t seem to ever end. Just to make sure I’ve tried it again 10 minutes ago.
Command is hanging and the process uses 0% cpu. I’ll let it run just in case.

I initially knew it was hanging through doom command. When you do a doom
Install it works fine, it starts with « Waiting for 2000 async process » and the number
steadily goes down to zero.

Then after that, doing doom sync again with changes will display a
« Waiting for 200 async process » and the number never moves, not even by one.
From there I took one of the files and tried it with emacs -Q to reproduce.

De : Eli Zaretskii<mailto:eliz@gnu.org>
Envoyé le :samedi 6 mars 2021 19:34
À : Matt M<mailto:mmerino@outlook.fr>
Cc : 46972@debbugs.gnu.org<mailto:46972@debbugs.gnu.org>
Objet :Re: bug#46972: 28.0.50; [feature/native-comp] Emacs locks itself during native compilation because of permission denied

> From: Matt M <mmerino@outlook.fr>
> Date: Sat, 6 Mar 2021 17:23:34 +0000
>
> I'm on windows using latest native compilation. I noticed that when doing
> doom sync (I use Doom emacs) it would hang on the native compilation
> step. I looked at the opened emacs processes in the task manager
> during this bug to try to find a file with which I could make the bug
> trigger again. In the task manager I see that about 10 emacs processes
> are stuck on 10 files.
>
> Using this file: https://pastebin.com/z4wLheXa as
> emacs-async-comp-ox-ascii.el, I call the following command:
>
> emacs -Q --batch -l c:/Users/Matt/emacs-async-comp-ox-ascii.el
> > Compiling c:/Users/Matt/.emacs.d/.local/straight/build-28.0.50/org-mode/ox-ascii.el...
>
> It never finishes.

Does it really "never" finish, or does it just take a very long time?
How long did you wait for it to finish?


[-- Attachment #2: Type: text/html, Size: 3814 bytes --]

bug#46972: RE : bug#46972: 28.0.50; [feature/native-comp] Emacs locks itself during native compilation because of permission denied

[Eli Zaretskii]

> From: Matt M <mmerino@outlook.fr>
> CC: "46972@debbugs.gnu.org" <46972@debbugs.gnu.org>
> Date: Sat, 6 Mar 2021 18:52:51 +0000
> 
> It doesn’t seem to ever end. Just to make sure I’ve tried it again 10 minutes ago. 
> Command is hanging and the process uses 0% cpu. I’ll let it run just in case.

If it uses 0% CPU, then waiting won't help.

How many Emacs processes are running on the system?

bug#46972: RE : RE : bug#46972: 28.0.50; [feature/native-comp] Emacs locks itself during native compilation because of permission denied

[Matt M]

[-- Attachment #1: Type: text/plain, Size: 825 bytes --]

Exactly one, and in the « Command line » column I see it has the arguments I passed it so
that’s the one

De : Eli Zaretskii<mailto:eliz@gnu.org>
Envoyé le :samedi 6 mars 2021 20:01
À : Matt M<mailto:mmerino@outlook.fr>
Cc : 46972@debbugs.gnu.org<mailto:46972@debbugs.gnu.org>
Objet :Re: RE : bug#46972: 28.0.50; [feature/native-comp] Emacs locks itself during native compilation because of permission denied

> From: Matt M <mmerino@outlook.fr>
> CC: "46972@debbugs.gnu.org" <46972@debbugs.gnu.org>
> Date: Sat, 6 Mar 2021 18:52:51 +0000
>
> It doesn’t seem to ever end. Just to make sure I’ve tried it again 10 minutes ago.
> Command is hanging and the process uses 0% cpu. I’ll let it run just in case.

If it uses 0% CPU, then waiting won't help.

How many Emacs processes are running on the system?


[-- Attachment #2: Type: text/html, Size: 2501 bytes --]

bug#46972: RE : RE : bug#46972: 28.0.50; [feature/native-comp] Emacs locks itself during native compilation because of permission denied

[Eli Zaretskii]

> From: Matt M <mmerino@outlook.fr>
> CC: "46972@debbugs.gnu.org" <46972@debbugs.gnu.org>
> Date: Sat, 6 Mar 2021 19:02:46 +0000
> 
> Exactly one, and in the « Command line » column I see it has the arguments I passed it so
> that’s the one

Hmm...  Andrea, what could that Emacs wait for?  When compilation
signals the error described by Matt, why doesn't Emacs exit instead of
waiting for something?

bug#46972: RE : RE : bug#46972: 28.0.50; [feature/native-comp] Emacs locks itself during native compilation because of permission denied

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Matt M <mmerino@outlook.fr>
>> CC: "46972@debbugs.gnu.org" <46972@debbugs.gnu.org>
>> Date: Sat, 6 Mar 2021 19:02:46 +0000
>> 
>> Exactly one, and in the « Command line » column I see it has the arguments I passed it so
>> that’s the one
>
> Hmm...  Andrea, what could that Emacs wait for?  When compilation
> signals the error described by Matt, why doesn't Emacs exit instead of
> waiting for something?

I *think* that what might be going on here is that Emacs is trying to
delete a stale .eln file through `comp-delete-or-replace-file'.  There
we have some Windows specific code that I guess might loop forever.

I guess somebody observing this issue will have confirm or refut this
suspect debugging this function.

  Andrea

bug#46972: RE : RE : bug#46972: 28.0.50; [feature/native-comp] Emacs locks itself during native compilation because of permission denied

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: Matt M <mmerino@outlook.fr>, 46972@debbugs.gnu.org
> Date: Sat, 06 Mar 2021 20:14:12 +0000
> 
> > Hmm...  Andrea, what could that Emacs wait for?  When compilation
> > signals the error described by Matt, why doesn't Emacs exit instead of
> > waiting for something?
> 
> I *think* that what might be going on here is that Emacs is trying to
> delete a stale .eln file through `comp-delete-or-replace-file'.  There
> we have some Windows specific code that I guess might loop forever.
> 
> I guess somebody observing this issue will have confirm or refut this
> suspect debugging this function.

OK, I will try to reproduce this and look into the cause(s).

bug#46972: RE : RE : RE : bug#46972: 28.0.50; [feature/native-comp] Emacs locks itself during native compilation because of permission denied

[Matt M]

[-- Attachment #1: Type: text/plain, Size: 1991 bytes --]

The Following patch seems to fix my problem :
--- a/lisp/emacs-lisp/comp.el
+++ b/lisp/emacs-lisp/comp.el
@@ -3778,13 +3778,7 @@ comp-delete-or-replace-file
          (while
              (condition-case _
                  (progn
-                   ;; oldfile maybe recreated by another Emacs in
-                   ;; between the following two rename-file calls
-                   (if (file-exists-p oldfile)
-                       (rename-file oldfile (make-temp-file-internal
-                                             (file-name-sans-extension oldfile)
-                                             nil ".eln.old" nil)
-                                    t))
+                   (delete-file oldfile)
                    (when newfile
                      (rename-file newfile oldfile nil))
                    ;; Keep on trying.
Changed the call to rename-file to delete-file.

De : Eli Zaretskii<mailto:eliz@gnu.org>
Envoyé le :samedi 6 mars 2021 21:16
À : Andrea Corallo<mailto:akrl@sdf.org>
Cc : mmerino@outlook.fr<mailto:mmerino@outlook.fr>; 46972@debbugs.gnu.org<mailto:46972@debbugs.gnu.org>
Objet :Re: RE : RE : bug#46972: 28.0.50; [feature/native-comp] Emacs locks itself during native compilation because of permission denied

> From: Andrea Corallo <akrl@sdf.org>
> Cc: Matt M <mmerino@outlook.fr>, 46972@debbugs.gnu.org
> Date: Sat, 06 Mar 2021 20:14:12 +0000
>
> > Hmm...  Andrea, what could that Emacs wait for?  When compilation
> > signals the error described by Matt, why doesn't Emacs exit instead of
> > waiting for something?
>
> I *think* that what might be going on here is that Emacs is trying to
> delete a stale .eln file through `comp-delete-or-replace-file'.  There
> we have some Windows specific code that I guess might loop forever.
>
> I guess somebody observing this issue will have confirm or refut this
> suspect debugging this function.

OK, I will try to reproduce this and look into the cause(s).


[-- Attachment #2: Type: text/html, Size: 7132 bytes --]

bug#46972: RE : RE : RE : bug#46972: 28.0.50; [feature/native-comp] Emacs locks itself during native compilation because of permission denied

[Eli Zaretskii]

> From: Matt M <mmerino@outlook.fr>
> CC: "46972@debbugs.gnu.org" <46972@debbugs.gnu.org>
> Date: Fri, 12 Mar 2021 00:26:18 +0000
> 
> The Following patch seems to fix my problem :
> --- a/lisp/emacs-lisp/comp.el
> +++ b/lisp/emacs-lisp/comp.el
> @@ -3778,13 +3778,7 @@ comp-delete-or-replace-file
>           (while
>               (condition-case _
>                   (progn
> -                   ;; oldfile maybe recreated by another Emacs in
> -                   ;; between the following two rename-file calls
> -                   (if (file-exists-p oldfile)
> -                       (rename-file oldfile (make-temp-file-internal
> -                                             (file-name-sans-extension oldfile)
> -                                             nil ".eln.old" nil)
> -                                    t))
> +                   (delete-file oldfile)
>                     (when newfile
>                       (rename-file newfile oldfile nil))
>                     ;; Keep on trying.
> Changed the call to rename-file to delete-file.

Thanks, but I don't think we can use this solution, due to the reason
explained in the comment there.  (You can, of course, use it locally,
if the danger described there is not relevant for your usage.)

I apologize that I didn't yet have time to recreate the problem you
described and look into it: the native-comp branch is still not stable
enough for me to try such non-trivial situations.  I didn't forget,
though.

bug#46972: RE : RE : RE : RE : bug#46972: 28.0.50; [feature/native-comp] Emacs locks itself during native compilation because of permission denied

[Matt M]

[-- Attachment #1: Type: text/plain, Size: 2612 bytes --]

If I understand correctly, this is what happens currently :

  1.  Rename oldfile to new temporary name
  2.  Rename newfile to oldfile
  3.  If rename in 2) failed because oldfile has been recreated, go back to 1)
With my change it becomes :

  1.  Delete oldfile
  2.  Rename newfile to oldfile
  3.  If rename in 2) failed because oldfile has been recreated, go back to 1)

The only difference being that oldfile is deleted instead of keeping an .eln.old file.
In the non Windows version of the code no .eln.old file is created anyway.

The problem on Windows is that the oldfile could reappear and block step 2),
so the while loop is necessary, but I keep it in my proposed change.

Or maybe I misunderstood entirely
De : Eli Zaretskii<mailto:eliz@gnu.org>
Envoyé le :vendredi 12 mars 2021 08:36
À : Matt M<mailto:MMERINO@OUTLOOK.FR>
Cc : akrl@sdf.org<mailto:akrl@sdf.org>; 46972@debbugs.gnu.org<mailto:46972@debbugs.gnu.org>
Objet :Re: RE : RE : RE : bug#46972: 28.0.50; [feature/native-comp] Emacs locks itself during native compilation because of permission denied

> From: Matt M <mmerino@outlook.fr>
> CC: "46972@debbugs.gnu.org" <46972@debbugs.gnu.org>
> Date: Fri, 12 Mar 2021 00:26:18 +0000
>
> The Following patch seems to fix my problem :
> --- a/lisp/emacs-lisp/comp.el
> +++ b/lisp/emacs-lisp/comp.el
> @@ -3778,13 +3778,7 @@ comp-delete-or-replace-file
>           (while
>               (condition-case _
>                   (progn
> -                   ;; oldfile maybe recreated by another Emacs in
> -                   ;; between the following two rename-file calls
> -                   (if (file-exists-p oldfile)
> -                       (rename-file oldfile (make-temp-file-internal
> -                                             (file-name-sans-extension oldfile)
> -                                             nil ".eln.old" nil)
> -                                    t))
> +                   (delete-file oldfile)
>                     (when newfile
>                       (rename-file newfile oldfile nil))
>                     ;; Keep on trying.
> Changed the call to rename-file to delete-file.

Thanks, but I don't think we can use this solution, due to the reason
explained in the comment there.  (You can, of course, use it locally,
if the danger described there is not relevant for your usage.)

I apologize that I didn't yet have time to recreate the problem you
described and look into it: the native-comp branch is still not stable
enough for me to try such non-trivial situations.  I didn't forget,
though.


[-- Attachment #2: Type: text/html, Size: 9816 bytes --]

bug#46972: RE : RE : RE : RE : bug#46972: 28.0.50; [feature/native-comp] Emacs locks itself during native compilation because of permission denied

[Eli Zaretskii]

> From: Matt M <mmerino@outlook.fr>
> CC: "akrl@sdf.org" <akrl@sdf.org>, "46972@debbugs.gnu.org"
> 	<46972@debbugs.gnu.org>
> Date: Fri, 12 Mar 2021 11:37:22 +0000
> 
> If I understand correctly, this is what happens currently :
> 
> 1 Rename oldfile to new temporary name
> 2 Rename newfile to oldfile
> 3 If rename in 2) failed because oldfile has been recreated, go back to 1)

Why would oldfile be recreated? which code recreates it?

bug#46972: RE : RE : RE : RE : RE : bug#46972: 28.0.50; [feature/native-comp] Emacs locks itself during native compilation because of permission denied

[Matt M]

[-- Attachment #1.1: Type: text/plain, Size: 1033 bytes --]

;; oldfile maybe recreated by another Emacs in
;; between the following two rename-file calls
I don’t know what code but the comment says that it can be. This
seems to be unique to Windows for some reason but I don’t know why.


De : Eli Zaretskii <eliz@gnu.org>
Envoyé : Friday, March 12, 2021 1:45:46 PM
À : Matt M <mmerino@outlook.fr>
Cc : akrl@sdf.org <akrl@sdf.org>; 46972@debbugs.gnu.org <46972@debbugs.gnu.org>
Objet : Re: RE : RE : RE : RE : bug#46972: 28.0.50; [feature/native-comp] Emacs locks itself during native compilation because of permission denied

> From: Matt M <mmerino@outlook.fr>
> CC: "akrl@sdf.org" <akrl@sdf.org>, "46972@debbugs.gnu.org"
>        <46972@debbugs.gnu.org>
> Date: Fri, 12 Mar 2021 11:37:22 +0000
>
> If I understand correctly, this is what happens currently :
>
> 1 Rename oldfile to new temporary name
> 2 Rename newfile to oldfile
> 3 If rename in 2) failed because oldfile has been recreated, go back to 1)

Why would oldfile be recreated? which code recreates it?

[-- Attachment #1.2: Type: text/html, Size: 3013 bytes --]

[-- Attachment #2: F038E641A1254CD3B029465A8B580B79.png --]
[-- Type: image/png, Size: 144 bytes --]

bug#46972: 28.0.50; [feature/native-comp] Emacs locks itself during native compilation because of permission denied

[Eli Zaretskii]

> From: Matt M <mmerino@outlook.fr>
> Date: Sat, 6 Mar 2021 17:23:34 +0000
> 
> I'm on windows using latest native compilation. I noticed that when doing
> doom sync (I use Doom emacs) it would hang on the native compilation
> step. I looked at the opened emacs processes in the task manager
> during this bug to try to find a file with which I could make the bug
> trigger again. In the task manager I see that about 10 emacs processes
> are stuck on 10 files.
> 
> Using this file: https://pastebin.com/z4wLheXa as
> emacs-async-comp-ox-ascii.el, I call the following command:
> 
> emacs -Q --batch -l c:/Users/Matt/emacs-async-comp-ox-ascii.el
> > Compiling c:/Users/Matt/.emacs.d/.local/straight/build-28.0.50/org-mode/ox-ascii.el...
> 
> It never finishes. If I interrupt the process I get the following
> backtrace:
> 
> Debugger entered--Lisp error: (file-error "Removing old name" "Permission denied" "c:/Users/Matt/.emacs.d/.local/cache/eln/28.0.50-4c...")
>   kill-emacs(t)
>   command-line()
>   normal-top-level()
> 
> Removing old name: Permission denied, c:/Users/Matt/.emacs.d/.local/cache/eln/28.0.50-4c6fa13f/ox-ascii-dcca1ba0-825ea35cp6iXUJ.eln.old
> Debugger entered--Lisp error: (file-error "Removing old name" "Permission denied" "c:/Users/Matt/.emacs.d/.local/cache/eln/28.0.50-4c...")
>   command-error-default-function((file-error "Removing old name" "Permission denied" "c:/Users/Matt/.emacs.d/.local/cache/eln/28.0.50-4c...") "" debug)
>   apply(command-error-default-function ((file-error "Removing old name" "Permission denied" "c:/Users/Matt/.emacs.d/.local/cache/eln/28.0.50-4c...") "" debug))
>   #f(advice-wrapper :after command-error-default-function help-command-error-confusable-suggestions)((file-error "Removing old name" "Permission denied" "c:/Users/Matt/.emacs.d/.local/cache/eln/28.0.50-4c...") "" debug)
> 
> It renamed a file from file.eln to file.eln.old, then tries to delete
> the file.eln.old but gets a Permission denied error.
> 
> If I start the same command again without interrupting, and try to
> delete the file.eln.old manually, I get the error:
> Can't remove file because the file is opened in emacs.exe
> 
> And the only emacs.exe process runnin on the computer is the one doing
> the native compilation, the one which created the file.eln.old.
> 
> So that seems to indicate that the emacs process that renames the file
> doesn't have the permission to remove the same file because of it being
> opened in itself.

I'm trying to understand what's going on in your case, and I'm quite
confused.  Can you help me understand what happens there?

The error message quoted above come from the delete-file call.
However, the only place where we call delete-file in
comp-delete-or-replace-file is here:

         (ignore-errors (delete-file oldfile))

and that ignores any errors.  So how come this still signals an error
in your case?  Likewise, I don't understand how replacing rename-file
with delete-file fixes this problem.

Moreover, I actually don't think these error messages are related to
what really happens during compilation, I think they are related to
the fact that you "interrupted the process" (how did you do that,
btw?).  Because the backtrace shows you invoked kill-emacs
interactively, and that triggered the error.  When Emacs exits, it
attempts to clean up *.eln.old files left behind.

Can you show the list of .eln.old files in the relevant directories
that are left when the compilation is stuck?  And also, can you try
figuring out whether any of those .eln.old files are open in any of
the running Emacs processes?  One way of finding this out is by using
the Process Explorer from SysInternals: it can show all the files that
a given process has open.

Andrea, it looks like the inner loop in eln_load_path_final_clean_up
isn't protected against errors?  IOW, if Fdelete_file signals an
error, it won't be caught, is that right?

bug#46972: 28.0.50; [feature/native-comp] Emacs locks itself during native compilation because of permission denied

[Eli Zaretskii]

> Date: Sun, 14 Mar 2021 12:51:06 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 46972@debbugs.gnu.org
> 
> Andrea, it looks like the inner loop in eln_load_path_final_clean_up
> isn't protected against errors?  IOW, if Fdelete_file signals an
> error, it won't be caught, is that right?

I've just had a similar situation myself, so I fixed the problem, I
hope.  Please try the latest native-comp branch and see if your
problem is solved.

bug#46972: RE : bug#46972: 28.0.50; [feature/native-comp] Emacs locks itself during native compilation because of permission denied

[Matt M]

[-- Attachment #1: Type: text/plain, Size: 849 bytes --]

I just tested it and the problem doesn’t happen anymore. Thanks a lot !

De : Eli Zaretskii<mailto:eliz@gnu.org>
Envoyé le :dimanche 14 mars 2021 14:38
À : mmerino@outlook.fr<mailto:mmerino@outlook.fr>
Cc : 46972@debbugs.gnu.org<mailto:46972@debbugs.gnu.org>
Objet :Re: bug#46972: 28.0.50; [feature/native-comp] Emacs locks itself during native compilation because of permission denied

> Date: Sun, 14 Mar 2021 12:51:06 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 46972@debbugs.gnu.org
>
> Andrea, it looks like the inner loop in eln_load_path_final_clean_up
> isn't protected against errors?  IOW, if Fdelete_file signals an
> error, it won't be caught, is that right?

I've just had a similar situation myself, so I fixed the problem, I
hope.  Please try the latest native-comp branch and see if your
problem is solved.


[-- Attachment #2: Type: text/html, Size: 2489 bytes --]

bug#46972: RE : bug#46972: 28.0.50; [feature/native-comp] Emacs locks itself during native compilation because of permission denied

[Eli Zaretskii]

> From: Matt M <mmerino@outlook.fr>
> CC: "46972@debbugs.gnu.org" <46972@debbugs.gnu.org>
> Date: Sun, 14 Mar 2021 14:07:13 +0000
> 
> I just tested it and the problem doesn’t happen anymore. Thanks a lot !

OK, thanks, I'm therefore closing this bug report.