bug#23692: 25.1.50; Package.el gets confused if file has no autoloads (could be a bug in update-directory-autoloads)

bug#23692: 25.1.50; Package.el gets confused if file has no autoloads (could be a bug in update-directory-autoloads)

[Clément Pit--Claudel]

[-- Attachment #1.1: Type: text/plain, Size: 2069 bytes --]

When the a single-package file has no autoloads, the call to update-directory-autoloads leaves the generated autoload files modified, but unsaved. This causes package.el to prompt the user about killing a modified file.

Small repro: paste the following in *scratch* and run package-install-from-buffer.


;;; test-package.el --- blah

;; Author: Me
;; Version: 1.0

(provide 'test-package)
;;; test-package.el ends here


This will show the following message: "Buffer test-package-autoloads.el modified; kill anyway? (yes or no)"

The problem happens when package.el reaches the following bit:

(defun package-generate-autoloads (name pkg-dir)
  (let* (...)
    ...
    (let ((buf (find-buffer-visiting generated-autoload-file)))
      (when buf (kill-buffer buf))) ;; <-- HERE
    ...))

The particular path that leads to an error condition is the following: inside of update-directory-autoloads, we see this:

      ;; Don't modify the file if its content has not been changed, so `make'
      ;; dependencies don't trigger unnecessarily.
      (when changed
        (let ((version-control 'never))
          (save-buffer)))

The flag `changed` is only set to `t` if files autoloads are found. However, file without autoloads also get added to the generated autoloads file:

    ;;;### (autoloads nil nil (some file names here) (0 0 0
    ;;;;;;  0))

I'm not sure which part of this is a bug: package.el assuming that update-directory-autoloads always saves the file after modifying it (which seems false), or update-directory-autoloads not saving the file. In any case, it's probably a misfeature that package.el prompts the user about killing a modified autoloads file as part of the installation process.

Thanks!
Clément.

In GNU Emacs 25.1.50.1 (x86_64-unknown-linux-gnu, GTK+ Version 3.10.8)
 of 2016-05-28 built on clem-w50-mint
Repository revision: 8377f2ed8aad863d6c59987b7b53fbceee46881e
Windowing system distributor 'The X.Org Foundation', version 11.0.11501000
System Description:	Linux Mint 17.3 Rosa


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

bug#23692: 25.1.50; Package.el gets confused if file has no autoloads (could be a bug in update-directory-autoloads)

[Glenn Morris]

Clément Pit--Claudel wrote:

> [...] the call to update-directory-autoloads leaves the generated
> autoload files modified, but unsaved.

Due to recent e971ce6de27f. Seems like an update-directory-autoloads bug
to me.

bug#23692: 25.1.50; Package.el gets confused if file has no autoloads (could be a bug in update-directory-autoloads)

[Clément Pit--Claudel]

[-- Attachment #1.1: Type: text/plain, Size: 335 bytes --]

On 2016-06-03 18:50, Glenn Morris wrote:
> Clément Pit--Claudel wrote:
> 
>> [...] the call to update-directory-autoloads leaves the generated
>> autoload files modified, but unsaved.
> 
> Due to recent e971ce6de27f. Seems like an update-directory-autoloads bug
> to me.

Brilliant, thanks for spotting this so quickly :)


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

bug#23692: 25.1.50; Package.el gets confused if file has no autoloads (could be a bug in update-directory-autoloads)

[Glenn Morris]

Stefan, FYI:

Glenn Morris wrote:

> Clément Pit--Claudel wrote:

>> When the a single-package file has no autoloads, the call to
>> update-directory-autoloads leaves the generated autoload files
>> modified, but unsaved. This causes package.el to prompt the user
>> about killing a modified file.
>
> Due to recent e971ce6de27f. Seems like an update-directory-autoloads
> bug to me.

bug#23692: 25.1.50; Package.el gets confused if file has no autoloads (could be a bug in update-directory-autoloads)

[Stefan Monnier]

>>> When the a single-package file has no autoloads, the call to
>>> update-directory-autoloads leaves the generated autoload files
>>> modified, but unsaved. This causes package.el to prompt the user
>>> about killing a modified file.

Does the patch below fix the problem for you?


        Stefan


diff --git a/lisp/emacs-lisp/autoload.el b/lisp/emacs-lisp/autoload.el
index 6473e31..9cf96fd 100644
--- a/lisp/emacs-lisp/autoload.el
+++ b/lisp/emacs-lisp/autoload.el
@@ -1111,7 +1111,8 @@ write its autoloads into the specified file instead."
 
       ;; Don't modify the file if its content has not been changed, so `make'
       ;; dependencies don't trigger unnecessarily.
-      (when changed
+      (if (not changed)
+          (set-buffer-modified-p nil)
         (let ((version-control 'never))
           (save-buffer)))
 

bug#23692: 25.1.50; Package.el gets confused if file has no autoloads (could be a bug in update-directory-autoloads)

[Glenn Morris]

Stefan Monnier wrote:

> -      (when changed
> +      (if (not changed)
> +          (set-buffer-modified-p nil)

I guess that will work. Some vague thoughts:

i) I guess now autoload will sometimes not create any output file.
Does that matter?

ii) I wonder what this means for reproducibility of the loaddefs file.
IIRC When building Emacs itself, the loaddefs file can change if you:
make -j8 bootstrap; make -C lisp autoloads
due to changes in "files with no autoloads" section owing to generated
files being created in parallel with loaddefs.el (this is really a
failure in dependency tracking for loaddefs, but it's harmless).
But it does finally converge to a fixed state. Will it still do so if
the loaddefs file sometimes isn't written?

iii) Would applying a move-if-change style logic be better?

bug#23692: 25.1.50; Package.el gets confused if file has no autoloads (could be a bug in update-directory-autoloads)

[Clément Pit--Claudel]

[-- Attachment #1.1: Type: text/plain, Size: 982 bytes --]

Yup, works for me!

On 2016-06-08 21:47, Stefan Monnier wrote:
>>>> When the a single-package file has no autoloads, the call to
>>>> update-directory-autoloads leaves the generated autoload files
>>>> modified, but unsaved. This causes package.el to prompt the user
>>>> about killing a modified file.
> 
> Does the patch below fix the problem for you?
> 
> 
>         Stefan
> 
> 
> diff --git a/lisp/emacs-lisp/autoload.el b/lisp/emacs-lisp/autoload.el
> index 6473e31..9cf96fd 100644
> --- a/lisp/emacs-lisp/autoload.el
> +++ b/lisp/emacs-lisp/autoload.el
> @@ -1111,7 +1111,8 @@ write its autoloads into the specified file instead."
>  
>        ;; Don't modify the file if its content has not been changed, so `make'
>        ;; dependencies don't trigger unnecessarily.
> -      (when changed
> +      (if (not changed)
> +          (set-buffer-modified-p nil)
>          (let ((version-control 'never))
>            (save-buffer)))
>  
> 
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

bug#23692: 25.1.50; Package.el gets confused if file has no autoloads (could be a bug in update-directory-autoloads)

[Stefan Monnier]

> i) I guess now autoload will sometimes not create any output file.

I think this shouldn't happen.  Both in the sense that it would be
a bug, and that my understanding of the code makes me think it
won't happen.  We create the file before we even try to update it.

> ii) I wonder what this means for reproducibility of the loaddefs file.

Good question.

> iii) Would applying a move-if-change style logic be better?

Could be, indeed.  My only motivation for the original patch was to
avoid repeatedly re-dumping Emacs even when no significant change had
happened.  I wrote the patch years ago, and at that point the
time-stamps were always different, so the buffer's content was
definitely changed, tho in ways which had no impact on the dump, so
"move-if-changed" was not an option.
Now that we avoid time-stamps maybe we can actually compare the buffer's
content with the file and skip saving if it's identical.


        Stefan

bug#23692: 25.1.50; Package.el gets confused if file has no autoloads (could be a bug in update-directory-autoloads)

[Eli Zaretskii]

> From: Stefan Monnier <monnier@IRO.UMontreal.CA>
> Date: Mon, 13 Jun 2016 20:40:24 -0400
> Cc: Clément Pit--Claudel <clement.pitclaudel@live.com>,
>  23692@debbugs.gnu.org
> 
> > i) I guess now autoload will sometimes not create any output file.
> 
> I think this shouldn't happen.  Both in the sense that it would be
> a bug, and that my understanding of the code makes me think it
> won't happen.  We create the file before we even try to update it.
> 
> > ii) I wonder what this means for reproducibility of the loaddefs file.
> 
> Good question.
> 
> > iii) Would applying a move-if-change style logic be better?
> 
> Could be, indeed.  My only motivation for the original patch was to
> avoid repeatedly re-dumping Emacs even when no significant change had
> happened.  I wrote the patch years ago, and at that point the
> time-stamps were always different, so the buffer's content was
> definitely changed, tho in ways which had no impact on the dump, so
> "move-if-changed" was not an option.
> Now that we avoid time-stamps maybe we can actually compare the buffer's
> content with the file and skip saving if it's identical.

Stefan, you didn't commit your patch, please do (it should go to the
master branch).

TIA

bug#23692: 25.1.50; Package.el gets confused if file has no autoloads (could be a bug in update-directory-autoloads)

[Stefan Monnier]

> Stefan, you didn't commit your patch, please do (it should go to the
> master branch).

Done,


        Stefan

bug#23692: 25.1.50; Package.el gets confused if file has no autoloads (could be a bug in update-directory-autoloads)

[Eli Zaretskii]

> From: Stefan Monnier <monnier@IRO.UMontreal.CA>
> Cc: 23692-done@debbugs.gnu.org, rgm@gnu.org, clement.pitclaudel@live.com
> Date: Tue, 11 Oct 2016 15:56:35 -0400
> 
> > Stefan, you didn't commit your patch, please do (it should go to the
> > master branch).
> 
> Done,

Thanks.