bug#61289: 30.0.50; Cannot reinstall language grammar after running `treesit-ready-p'

bug#61289: 30.0.50; Cannot reinstall language grammar after running `treesit-ready-p'

[牟 桐]

[-- Attachment #1: Type: text/plain, Size: 4292 bytes --]



Hi there,

I'm using Windows, so I'll use `libtree-sitter-c.dll` as an example:

1. `emacs -Q` to start a vanilla Emacs.

Run `M-x treesit-install-language-grammar`, then install the c
parser to `~/.emacs.d/tree-sitter`. If there is already a
`libtree-sitter-c.dll` there, the new installed library will
overwrite the old library.

2. Execute `(treesit-ready-p 'c)`, which returns t.

3. Run `M-x treesit-install-language-grammar` again, there will be a
warning:

```
⛔ Warning (treesit): Error encountered when installing language grammar: (file-error Copying file Operation not permitted c:/Users/redacted/AppData/Local/Temp/treesit-workdir5HPGoS/repo/src/libtree-sitter-c.dll c:/Users/redacted/.emacs.d/tree-sitter/libtree-sitter-c.dll)
```

I don't know whether this is a bug or not, since I think it's reasonable
to make the already used library not be able to be overwritten.

But someone reminded me that `treesit-ready-p` will be executed when you
load the `xxx-ts-mode`. If the execution of `treesit-ready-p` made the
`treesit-install-language-grammar` unusable, it means the design of that
function has a little problem.

---

dalu


In GNU Emacs 30.0.50 (build 7, x86_64-w64-mingw32) of 2023-02-03 built
on DALUW
Repository revision: ac7ec87a7a0db887e4ae7fe9005aea517958b778
Repository branch: master
Windowing system distributor 'Microsoft Corp.', version 10.0.22621
System Description: Microsoft Windows 10 Home China (v10.0.2009.22621.1194)

Configured using:
'configure --prefix=/c/emacs/emacs-git --without-dbus --without-pop
--with-native-compilation=aot'

Configured features:
ACL GIF GMP GNUTLS HARFBUZZ JPEG JSON LCMS2 LIBXML2 MODULES NATIVE_COMP
NOTIFY W32NOTIFY PDUMPER PNG RSVG SOUND SQLITE3 THREADS TIFF
TOOLKIT_SCROLL_BARS TREE_SITTER WEBP XPM ZLIB

Important settings:
  value of $LANG: CHS
  locale-coding-system: cp65001

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message mailcap yank-media puny dired
dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068
epg-config gnus-util text-property-search time-date mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail
rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils vc-git
diff-mode easy-mmode vc-dispatcher treesit cl-loaddefs comp comp-cstr
warnings icons subr-x rx cl-seq cl-macs gv cl-extra help-mode bytecomp
byte-compile cl-lib china-util rmc iso-transl tooltip cconv eldoc paren
electric uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel
dos-w32 ls-lisp disp-table term/w32-win w32-win w32-vars term/common-win
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace
newcomment text-mode lisp-mode prog-mode register page tab-bar menu-bar
rfn-eshadow isearch easymenu timer select scroll-bar mouse jit-lock
font-lock syntax font-core term/tty-colors frame minibuffer nadvice seq
simple cl-generic indonesian philippine cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese composite emoji-zwj charscript charprop case-table
epa-hook jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button
loaddefs theme-loaddefs faces cus-face macroexp files window
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget keymap hashtable-print-readable backquote threads
w32notify w32 lcms2 multi-tty make-network-process native-compile emacs)

Memory information:
((conses 16 87683 14604)
(symbols 48 7929 0)
(strings 32 22771 2959)
(string-bytes 1 688676)
(vectors 16 16331)
(vector-slots 8 340811 17500)
(floats 8 48 160)
(intervals 56 406 0)
(buffers 984 13))

[-- Attachment #2: Type: text/html, Size: 12151 bytes --]

bug#61289: 30.0.50; Cannot reinstall language grammar after running `treesit-ready-p'

[Eli Zaretskii]

> From: 牟 桐 <mou.tong@outlook.com>
> Date: Sat, 4 Feb 2023 12:59:53 +0000
> 
> I'm using Windows, so I'll use `libtree-sitter-c.dll` as an example:
> 
> 1. `emacs -Q` to start a vanilla Emacs.
> 
> Run `M-x treesit-install-language-grammar`, then install the c
> parser to `~/.emacs.d/tree-sitter`. If there is already a
> `libtree-sitter-c.dll` there, the new installed library will
> overwrite the old library.
> 
> 2. Execute `(treesit-ready-p 'c)`, which returns t.
> 
> 3. Run `M-x treesit-install-language-grammar` again, there will be a
> warning:
> 
> ```
> ⛔ Warning (treesit): Error encountered when installing language grammar: (file-error Copying file Operation not permitted c:/Users/redacted/AppData/Local/Temp/treesit-workdir5HPGoS/repo/src/libtree-sitter-c.dll c:/Users/redacted/.emacs.d/tree-sitter/libtree-sitter-c.dll)
> ```
> 
> I don't know whether this is a bug or not, since I think it's reasonable
> to make the already used library not be able to be overwritten.
> 
> But someone reminded me that `treesit-ready-p` will be executed when you
> load the `xxx-ts-mode`. If the execution of `treesit-ready-p` made the
> `treesit-install-language-grammar` unusable, it means the design of that
> function has a little problem.

This is a basic issue on MS-Windows: it won't allow you to overwrite a
DLL that is being used by some program.

Can you try the patch below and see if it solves the problem?

diff --git a/lisp/treesit.el b/lisp/treesit.el
index 948016d..7e31da9 100644
--- a/lisp/treesit.el
+++ b/lisp/treesit.el
@@ -2884,7 +2884,17 @@ treesit--install-language-grammar-1
           ;; Copy out.
           (unless (file-exists-p out-dir)
             (make-directory out-dir t))
-          (copy-file lib-name (file-name-as-directory out-dir) t t)
+          (let* ((library-fname (expand-file-name lib-name out-dir))
+                 (old-fname (concat library-fname ".old")))
+            ;; Rename the existing shared library, if any, then
+            ;; install the new one, and try deleting the old one.
+            ;; This is for Windows systems, where we cannot simply
+            ;; overwrite a DLL that is being used.
+            (if (file-exists-p library-fname)
+                (rename-file library-fname old-fname t))
+            (copy-file lib-name (file-name-as-directory out-dir) t t)
+            ;; Ignore errors, in case the old version is still used.
+            (ignore-errors (delete-file old-fname)))
           (message "Library installed to %s/%s" out-dir lib-name))
       (when (file-exists-p workdir)
         (delete-directory workdir t)))))

bug#61289: 回复: bug#61289: 30.0.50; Cannot reinstall language grammar after running `treesit-ready-p'

[牟 桐]

[-- Attachment #1: Type: text/plain, Size: 4749 bytes --]

Yes, after applying your patch, I believe this problem has been solved.

But I noticed the `libtree-sitter-xxx.dll.old` in the tree-sitter dir
hadn't been delelted.

If I remove the `ignore-errors` function in your patch, which seems like
this (sorry, I'm not a programmer and don't know how to use diff):

-          (copy-file lib-name (file-name-as-directory out-dir) t t)
+          (let* ((library-fname (expand-file-name lib-name out-dir))
+                 (old-fname (concat library-fname ".old")))
+            ;; Rename the existing shared library, if any, then
+            ;; install the new one, and try deleting the old one.
+            ;; This is for Windows systems, where we cannot simply
+            ;; overwrite a DLL that is being used.
+            (if (file-exists-p library-fname)
+                (rename-file library-fname old-fname t))
+            (copy-file lib-name (file-name-as-directory out-dir) t t)
+            ;; Ignore errors, in case the old version is still used.
+            (delete-file old-fname))

             ^
             I deleted the outside `ignore-errors` to know why the `xxx.old` hadn’t been deleted.

After applying the change, I got the same error as I mentioned before:

```
⛔ Warning (treesit): Error encountered when installing language grammar: (permission-denied Removing old name Permission denied c:/Users/redacted/.emacs.d/tree-sitter/libtree-sitter-c.dll.old)
```

Btw, can we make the variable ` treesit-language-source-alist`
customizable? (Use `defcustom` instead of `defvar`.) I'm using
`use-package` to manage my config. When I'm configuring treesit, I'd
like to use `:custom` instead of `:init` or `:config` since it can let
me set variable's value without writing `setq`.

发件人: Eli Zaretskii<mailto:eliz@gnu.org>
发送时间: 2023年2月5日 17:13
收件人: 牟 桐<mailto:mou.tong@outlook.com>
抄送: 61289@debbugs.gnu.org<mailto:61289@debbugs.gnu.org>
主题: Re: bug#61289: 30.0.50; Cannot reinstall language grammar after running `treesit-ready-p'

> From: 牟 桐 <mou.tong@outlook.com>
> Date: Sat, 4 Feb 2023 12:59:53 +0000
>
> I'm using Windows, so I'll use `libtree-sitter-c.dll` as an example:
>
> 1. `emacs -Q` to start a vanilla Emacs.
>
> Run `M-x treesit-install-language-grammar`, then install the c
> parser to `~/.emacs.d/tree-sitter`. If there is already a
> `libtree-sitter-c.dll` there, the new installed library will
> overwrite the old library.
>
> 2. Execute `(treesit-ready-p 'c)`, which returns t.
>
> 3. Run `M-x treesit-install-language-grammar` again, there will be a
> warning:
>
> ```
> ⛔ Warning (treesit): Error encountered when installing language grammar: (file-error Copying file Operation not permitted c:/Users/redacted/AppData/Local/Temp/treesit-workdir5HPGoS/repo/src/libtree-sitter-c.dll c:/Users/redacted/.emacs.d/tree-sitter/libtree-sitter-c.dll)
> ```
>
> I don't know whether this is a bug or not, since I think it's reasonable
> to make the already used library not be able to be overwritten.
>
> But someone reminded me that `treesit-ready-p` will be executed when you
> load the `xxx-ts-mode`. If the execution of `treesit-ready-p` made the
> `treesit-install-language-grammar` unusable, it means the design of that
> function has a little problem.

This is a basic issue on MS-Windows: it won't allow you to overwrite a
DLL that is being used by some program.

Can you try the patch below and see if it solves the problem?

diff --git a/lisp/treesit.el b/lisp/treesit.el
index 948016d..7e31da9 100644
--- a/lisp/treesit.el
+++ b/lisp/treesit.el
@@ -2884,7 +2884,17 @@ treesit--install-language-grammar-1
           ;; Copy out.
           (unless (file-exists-p out-dir)
             (make-directory out-dir t))
-          (copy-file lib-name (file-name-as-directory out-dir) t t)
+          (let* ((library-fname (expand-file-name lib-name out-dir))
+                 (old-fname (concat library-fname ".old")))
+            ;; Rename the existing shared library, if any, then
+            ;; install the new one, and try deleting the old one.
+            ;; This is for Windows systems, where we cannot simply
+            ;; overwrite a DLL that is being used.
+            (if (file-exists-p library-fname)
+                (rename-file library-fname old-fname t))
+            (copy-file lib-name (file-name-as-directory out-dir) t t)
+            ;; Ignore errors, in case the old version is still used.
+            (ignore-errors (delete-file old-fname)))
           (message "Library installed to %s/%s" out-dir lib-name))
       (when (file-exists-p workdir)
         (delete-directory workdir t)))))


[-- Attachment #2: Type: text/html, Size: 12220 bytes --]

bug#61289: 回复: bug#61289: 30.0.50; Cannot reinstall language grammar after running `treesit-ready-p'

[Eli Zaretskii]

> From: 牟 桐 <mou.tong@outlook.com>
> CC: "61289@debbugs.gnu.org" <61289@debbugs.gnu.org>
> Date: Sun, 5 Feb 2023 09:58:49 +0000
> 
> Yes, after applying your patch, I believe this problem has been solved.

Thanks.  I installed the change on the emacs-29 branch, and I'm
closing this bug.

> But I noticed the `libtree-sitter-xxx.dll.old` in the tree-sitter dir
> hadn't been delelted.

It cannot be deleted as long as the Emacs session which uses it is up
and running.  Windows doesn't let us delete a DLL that is being used,
that's why the patch renames it instead.

The next time you install a new version of the same DLL, the .old file
will be deleted, if it still exists.  Or you can delete it by hand.

> If I remove the `ignore-errors` function in your patch, which seems like
> this (sorry, I'm not a programmer and don't know how to use diff):
> 
> -          (copy-file lib-name (file-name-as-directory out-dir) t t)
> +          (let* ((library-fname (expand-file-name lib-name out-dir))
> +                 (old-fname (concat library-fname ".old")))
> +            ;; Rename the existing shared library, if any, then
> +            ;; install the new one, and try deleting the old one.
> +            ;; This is for Windows systems, where we cannot simply
> +            ;; overwrite a DLL that is being used.
> +            (if (file-exists-p library-fname)
> +                (rename-file library-fname old-fname t))
> +            (copy-file lib-name (file-name-as-directory out-dir) t t)
> +            ;; Ignore errors, in case the old version is still used.
> +            (delete-file old-fname))
> 
>              ^
>              I deleted the outside `ignore-errors` to know why the `xxx.old` hadn’t been deleted.
> 
> After applying the change, I got the same error as I mentioned before:
> 
> ```
> ⛔ Warning (treesit): Error encountered when installing language grammar: (permission-denied Removing old name Permission denied c:/Users/redacted/.emacs.d/tree-sitter/libtree-sitter-c.dll.old)
> ```

Of course: that's why the patch uses ignore-errors in the first
place.  The DLL cannot be deleted as long as it is used.

> Btw, can we make the variable ` treesit-language-source-alist`
> customizable? (Use `defcustom` instead of `defvar`.) I'm using
> `use-package` to manage my config. When I'm configuring treesit, I'd
> like to use `:custom` instead of `:init` or `:config` since it can let
> me set variable's value without writing `setq`.

This was considered, but we decided not to do that, for several good
reasons.  One reason is that Emacs itself changes the value, which is
not appropriate with defcustoms.

bug#61289: 回复: 回复: bug#61289: 30.0.50; Cannot reinstall language grammar after running `treesit-ready-p'

[牟 桐]

[-- Attachment #1: Type: text/plain, Size: 3706 bytes --]

> Thanks.  I installed the change on the emacs-29 branch, and I'm
> closing this bug.

I compiled a new Emacs based on emacs-29 branch, it works fine, thanks
for your patience. Hope it will be merged to master soon 😊

> It cannot be deleted as long as the Emacs session which uses it is up
> and running.  Windows doesn't let us delete a DLL that is being used,
> that's why the patch renames it instead.

I get it, thanks for the explanation.

> This was considered, but we decided not to do that, for several good
> reasons.  One reason is that Emacs itself changes the value, which is
> not appropriate with defcustoms.

Oh, it seems that I think too little.

---

发件人: Eli Zaretskii<mailto:eliz@gnu.org>
发送时间: 2023年2月5日 18:24
收件人: 牟 桐<mailto:mou.tong@outlook.com>
抄送: 61289-done@debbugs.gnu.org<mailto:61289-done@debbugs.gnu.org>
主题: Re: 回复: bug#61289: 30.0.50; Cannot reinstall language grammar after running `treesit-ready-p'

> From: 牟 桐 <mou.tong@outlook.com>
> CC: "61289@debbugs.gnu.org" <61289@debbugs.gnu.org>
> Date: Sun, 5 Feb 2023 09:58:49 +0000
>
> Yes, after applying your patch, I believe this problem has been solved.

Thanks.  I installed the change on the emacs-29 branch, and I'm
closing this bug.

> But I noticed the `libtree-sitter-xxx.dll.old` in the tree-sitter dir
> hadn't been delelted.

It cannot be deleted as long as the Emacs session which uses it is up
and running.  Windows doesn't let us delete a DLL that is being used,
that's why the patch renames it instead.

The next time you install a new version of the same DLL, the .old file
will be deleted, if it still exists.  Or you can delete it by hand.

> If I remove the `ignore-errors` function in your patch, which seems like
> this (sorry, I'm not a programmer and don't know how to use diff):
>
> -          (copy-file lib-name (file-name-as-directory out-dir) t t)
> +          (let* ((library-fname (expand-file-name lib-name out-dir))
> +                 (old-fname (concat library-fname ".old")))
> +            ;; Rename the existing shared library, if any, then
> +            ;; install the new one, and try deleting the old one.
> +            ;; This is for Windows systems, where we cannot simply
> +            ;; overwrite a DLL that is being used.
> +            (if (file-exists-p library-fname)
> +                (rename-file library-fname old-fname t))
> +            (copy-file lib-name (file-name-as-directory out-dir) t t)
> +            ;; Ignore errors, in case the old version is still used.
> +            (delete-file old-fname))
>
>              ^
>              I deleted the outside `ignore-errors` to know why the `xxx.old` hadn’t been deleted.
>
> After applying the change, I got the same error as I mentioned before:
>
> ```
> ⛔ Warning (treesit): Error encountered when installing language grammar: (permission-denied Removing old name Permission denied c:/Users/redacted/.emacs.d/tree-sitter/libtree-sitter-c.dll.old)
> ```

Of course: that's why the patch uses ignore-errors in the first
place.  The DLL cannot be deleted as long as it is used.

> Btw, can we make the variable ` treesit-language-source-alist`
> customizable? (Use `defcustom` instead of `defvar`.) I'm using
> `use-package` to manage my config. When I'm configuring treesit, I'd
> like to use `:custom` instead of `:init` or `:config` since it can let
> me set variable's value without writing `setq`.

This was considered, but we decided not to do that, for several good
reasons.  One reason is that Emacs itself changes the value, which is
not appropriate with defcustoms.


[-- Attachment #2: Type: text/html, Size: 9018 bytes --]