bug#25360: File mode specification errors during building

bug#25360: File mode specification errors during building

[Glenn Morris]

Package: emacs
Version: 26.0.50

In current master, the following errors occur during building:

  Converting cangjie-table.b5 to tsang-b5.el...
  File mode specification error: (wrong-type-argument stringp image-type-auto-detected-p)

(same for quick-b5.el, tsang-cns.el, quick-cns.el, PY.el, ZIRANMA.el)

  Converting CTLau.html to CTLau.el...
  File mode specification error: (void-function html-mode)

(same for CTLau-b5.el)


They don't actually seem to stop the build, but still they should not be
happening. (I would guess recent changes related to loaddefs generation
are the cause.)


See eg http://hydra.nixos.org/build/45973447/log/raw

bug#25360: File mode specification errors during building

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Date: Wed, 04 Jan 2017 15:28:26 -0500
> 
> In current master, the following errors occur during building:
> 
>   Converting cangjie-table.b5 to tsang-b5.el...
>   File mode specification error: (wrong-type-argument stringp image-type-auto-detected-p)
> 
> (same for quick-b5.el, tsang-cns.el, quick-cns.el, PY.el, ZIRANMA.el)
> 
>   Converting CTLau.html to CTLau.el...
>   File mode specification error: (void-function html-mode)
> 
> (same for CTLau-b5.el)
> 
> 
> They don't actually seem to stop the build, but still they should not be
> happening. (I would guess recent changes related to loaddefs generation
> are the cause.)
> 
> 
> See eg http://hydra.nixos.org/build/45973447/log/raw

Phillip, could you please look into this?

Thanks.

bug#25360: File mode specification errors during building

[Phillip Lord]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Glenn Morris <rgm@gnu.org>
>> Date: Wed, 04 Jan 2017 15:28:26 -0500
>> 
>> In current master, the following errors occur during building:
>> 
>>   Converting cangjie-table.b5 to tsang-b5.el...
>>   File mode specification error: (wrong-type-argument stringp image-type-auto-detected-p)
>> 
>> (same for quick-b5.el, tsang-cns.el, quick-cns.el, PY.el, ZIRANMA.el)
>> 
>>   Converting CTLau.html to CTLau.el...
>>   File mode specification error: (void-function html-mode)
>> 
>> (same for CTLau-b5.el)
>> 
>> 
>> They don't actually seem to stop the build, but still they should not be
>> happening. (I would guess recent changes related to loaddefs generation
>> are the cause.)
>> 
>> 
>> See eg http://hydra.nixos.org/build/45973447/log/raw
>
> Phillip, could you please look into this?


Yes, that's caused by me (although I don't understand how yet).

I will look at it.

Phil

bug#25360: File mode specification errors during building

[Phillip Lord]

Glenn Morris <rgm@gnu.org> writes:

> Package: emacs
> Version: 26.0.50
>
> In current master, the following errors occur during building:
>
>   Converting cangjie-table.b5 to tsang-b5.el...
>   File mode specification error: (wrong-type-argument stringp image-type-auto-detected-p)
>
> (same for quick-b5.el, tsang-cns.el, quick-cns.el, PY.el, ZIRANMA.el)
>
>   Converting CTLau.html to CTLau.el...
>   File mode specification error: (void-function html-mode)
>
> (same for CTLau-b5.el)
>
>
> They don't actually seem to stop the build, but still they should not be
> happening. (I would guess recent changes related to loaddefs generation
> are the cause.)

I do not quite understand why this is not working -- loaddefs.el has
been generated at this point. I still do not fully understand what is
happening during the dumping process, though, so I guess bootstrap-emacs
is different now than before.

Before I investigate further, two simple solutions leap to mind. One is
just to suppress the errors probably in "titdic-cnv.el".

The other would be just to stop generating CTLau.el and
tstang-b5.el. Neither of the files from which they are generated have
been materially changed in since 2001, and CTLau.html apparently does
not exist any more in its source location. Deleting them, and adding
CTLau.el as source would solve the problem and remove a step from the
build.

Thoughts?

Phil

bug#25360: File mode specification errors during building

[Eli Zaretskii]

> From: phillip.lord@russet.org.uk (Phillip Lord)
> Date: Tue, 10 Jan 2017 10:21:12 +0000
> Cc: 25360@debbugs.gnu.org
> 
> I do not quite understand why this is not working -- loaddefs.el has
> been generated at this point. I still do not fully understand what is
> happening during the dumping process, though, so I guess bootstrap-emacs
> is different now than before.

Can you describe what you did understand from looking into this issue?
E.g., which code emits the error and why?

> Before I investigate further, two simple solutions leap to mind. One is
> just to suppress the errors probably in "titdic-cnv.el".
> 
> The other would be just to stop generating CTLau.el and
> tstang-b5.el. Neither of the files from which they are generated have
> been materially changed in since 2001, and CTLau.html apparently does
> not exist any more in its source location. Deleting them, and adding
> CTLau.el as source would solve the problem and remove a step from the
> build.
> 
> Thoughts?

I think we should try to understand the problem before we consider
solutions.  In particular, the same problem could be at work in other
places, just without such a spectacular indications.

Thanks.

bug#25360: File mode specification errors during building

[Phillip Lord]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: phillip.lord@russet.org.uk (Phillip Lord)
>> Date: Tue, 10 Jan 2017 10:21:12 +0000
>> Cc: 25360@debbugs.gnu.org
>> 
>> I do not quite understand why this is not working -- loaddefs.el has
>> been generated at this point. I still do not fully understand what is
>> happening during the dumping process, though, so I guess bootstrap-emacs
>> is different now than before.
>
> Can you describe what you did understand from looking into this issue?
> E.g., which code emits the error and why?

One error is caused when bootstrap-emacs loads CTLau.html, when it's
running this code in leim.

misc_convert = $(AM_V_GEN)${RUN_EMACS} \
  -l titdic-cnv -f batch-miscdic-convert -dir ${leimdir}/quail

## CTLau.el, CTLau-b5.el.
${leimdir}/quail/CT%.el: ${srcdir}/MISC-DIC/CT%.html
	${misc_convert} $<

As far as I can tell it is happening because emacs tries to put
CTLau.html into html-mode. AFAICT, batch-miscdic-convert uses no
features of html-mode.

I've seen this sort of thing before; the deep problem is that
bootstrap-emacs obeys things like auto-mode-alist and .dir-locals.el,
when it probably doesn't really need to.

>> Before I investigate further, two simple solutions leap to mind. One is
>> just to suppress the errors probably in "titdic-cnv.el".
>> 
>> The other would be just to stop generating CTLau.el and
>> tstang-b5.el. Neither of the files from which they are generated have
>> been materially changed in since 2001, and CTLau.html apparently does
>> not exist any more in its source location. Deleting them, and adding
>> CTLau.el as source would solve the problem and remove a step from the
>> build.
>> 
>> Thoughts?
>
> I think we should try to understand the problem before we consider
> solutions.  In particular, the same problem could be at work in other
> places, just without such a spectacular indications.

True enough. Although, we don't have the problem anywhere else. Shortly
after this, emacs dumps fully, then the first version of bootstrap-emacs
is replaced with the fully dumped emacs. These errors will occur only
during a full bootstrap build I think.

Still, I'll plug at it a little more.

Phil

bug#25360: File mode specification errors during building

[Eli Zaretskii]

> From: phillip.lord@russet.org.uk (Phillip Lord)
> Cc: rgm@gnu.org,  25360@debbugs.gnu.org
> Date: Tue, 10 Jan 2017 18:40:28 +0000
> 
> One error is caused when bootstrap-emacs loads CTLau.html, when it's
> running this code in leim.
> 
> misc_convert = $(AM_V_GEN)${RUN_EMACS} \
>   -l titdic-cnv -f batch-miscdic-convert -dir ${leimdir}/quail
> 
> ## CTLau.el, CTLau-b5.el.
> ${leimdir}/quail/CT%.el: ${srcdir}/MISC-DIC/CT%.html
> 	${misc_convert} $<
> 
> As far as I can tell it is happening because emacs tries to put
> CTLau.html into html-mode. AFAICT, batch-miscdic-convert uses no
> features of html-mode.

Ok, and how does image-type-auto-detected-p enter this picture?

> Still, I'll plug at it a little more.

Thanks.

bug#25360: File mode specification errors during building

[Phillip Lord]

[-- Attachment #1: Type: text/plain, Size: 1168 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: phillip.lord@russet.org.uk (Phillip Lord)
>> Cc: rgm@gnu.org,  25360@debbugs.gnu.org
>> Date: Tue, 10 Jan 2017 18:40:28 +0000
>> 
>> One error is caused when bootstrap-emacs loads CTLau.html, when it's
>> running this code in leim.
>> 
>> misc_convert = $(AM_V_GEN)${RUN_EMACS} \
>>   -l titdic-cnv -f batch-miscdic-convert -dir ${leimdir}/quail
>> 
>> ## CTLau.el, CTLau-b5.el.
>> ${leimdir}/quail/CT%.el: ${srcdir}/MISC-DIC/CT%.html
>> 	${misc_convert} $<
>> 
>> As far as I can tell it is happening because emacs tries to put
>> CTLau.html into html-mode. AFAICT, batch-miscdic-convert uses no
>> features of html-mode.
>
> Ok, and how does image-type-auto-detected-p enter this picture?
>
>> Still, I'll plug at it a little more.
>
> Thanks.


So, the cause of the problem is this. bootstrap-emacs (nor emacs) does
not actually load loaddefs.el; it's just dumped with the binary. I was
recording autoloads until loaddefs.el was generated, but actually, I
need to record them till loaddefs.el is used, that is when emacs is
dumped.

The attached patch seems to address. Can you confirm, then I will apply
to master.



[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Record-autoloads-till-emacs-dump.patch --]
[-- Type: text/x-diff, Size: 7406 bytes --]

From 74539c444487094e45a0dd1b4cdb4553eb9cdfe1 Mon Sep 17 00:00:00 2001
From: Phillip Lord <phillip.lord@russet.org.uk>
Date: Fri, 13 Jan 2017 13:57:51 +0000
Subject: [PATCH] Record autoloads till emacs dump

* admin/ldefs-clean.el (ldefs-clean-up): Record autoloads till emacs dump
* lisp/ldefs-boot-auto.el (batch-byte-compile): Update

Previously, autoloads were collected till loaddefs.el was generated as
part of the build. However, bootstrap-emacs does not load
loaddefs (rather it is dumped), hence we must record autoloads until the
full emacs binary is dumped.
---
 admin/ldefs-clean.el    |  3 ++-
 lisp/ldefs-boot-auto.el | 34 ++++++++++++++++++++++++++++++++++
 2 files changed, 36 insertions(+), 1 deletion(-)

diff --git a/admin/ldefs-clean.el b/admin/ldefs-clean.el
index 6eabe57..808b537 100644
--- a/admin/ldefs-clean.el
+++ b/admin/ldefs-clean.el
@@ -39,7 +39,8 @@ ldefs-clean-up
   (goto-char (point-max))
   ;; We only need the autoloads up till loaddefs.el is
   ;; generated. After this, ldefs-boot.el is not needed
-  (search-backward "  GEN      loaddefs.el")
+  (search-backward "-l loadup dump")
+  (beginning-of-line)
   (delete-region (point) (point-max))
   (keep-lines "(autoload" (point-min) (point-max))
   (sort-lines nil (point-min) (point-max))
diff --git a/lisp/ldefs-boot-auto.el b/lisp/ldefs-boot-auto.el
index 914fec8..020c670 100644
--- a/lisp/ldefs-boot-auto.el
+++ b/lisp/ldefs-boot-auto.el
@@ -6,6 +6,8 @@
 (autoload 'add-change-log-entry "add-log" nil nil nil)
 (autoload 'add-log-current-defun "add-log" nil nil nil)
 (autoload 'batch-byte-compile "bytecomp" nil nil nil)
+(autoload 'batch-update-autoloads "autoload" nil nil nil)
+(autoload 'bounds-of-thing-at-point "thingatpt" nil nil nil)
 (autoload 'browse-url "browse-url" nil nil nil)
 (autoload 'buffer-face-mode "face-remap" nil nil nil)
 (autoload 'byte-compile "bytecomp" nil nil nil)
@@ -24,11 +26,16 @@
 (autoload 'compilation-mode "compile" nil nil nil)
 (autoload 'compilation-shell-minor-mode "compile" nil nil nil)
 (autoload 'compilation-start "compile" nil nil nil)
+(autoload 'completing-read-multiple "crm" nil nil nil)
+(autoload 'conf-mode "conf-mode" nil nil nil)
+(autoload 'create-glyph "disp-table" nil nil nil)
 (autoload 'create-image "image" nil nil nil)
+(autoload 'cursor-sensor-mode "cursor-sensor" nil nil nil)
 (autoload 'custom-save-all "cus-edit" nil nil nil)
 (autoload 'customize-face "cus-edit" nil nil nil)
 (autoload 'customize-group "cus-edit" nil nil nil)
 (autoload 'customize-option "cus-edit" nil nil nil)
+(autoload 'customize-push-and-save "cus-edit" nil nil nil)
 (autoload 'customize-set-variable "cus-edit" nil nil nil)
 (autoload 'debug "debug" nil nil nil)
 (autoload 'define-ccl-program "ccl" nil nil t)
@@ -36,6 +43,8 @@
 (autoload 'define-minor-mode "easy-mmode" nil nil t)
 (autoload 'delete-extract-rectangle "rect" nil nil nil)
 (autoload 'describe-char "descr-text" nil nil nil)
+(autoload 'describe-coding-system "mule-diag" nil nil nil)
+(autoload 'describe-display-table "disp-table" nil nil nil)
 (autoload 'describe-function "help-fns" nil nil nil)
 (autoload 'describe-function-1 "help-fns" nil nil nil)
 (autoload 'describe-package "package" nil nil nil)
@@ -43,11 +52,21 @@
 (autoload 'desktop-save "desktop" nil nil nil)
 (autoload 'diff-mode "diff-mode" nil nil nil)
 (autoload 'dired "dired" nil nil nil)
+(autoload 'dired-copy-file "dired-aux" nil nil nil)
+(autoload 'dired-goto-subdir "dired-aux" nil nil nil)
+(autoload 'dired-hide-subdir "dired-aux" nil nil nil)
+(autoload 'dired-insert-subdir "dired-aux" nil nil nil)
+(autoload 'dired-kill-subdir "dired-aux" nil nil nil)
+(autoload 'dired-mark-subdir-files "dired-aux" nil nil nil)
 (autoload 'dired-mode "dired" nil nil nil)
 (autoload 'dired-noselect "dired" nil nil nil)
+(autoload 'dired-query "dired-aux" nil nil nil)
+(autoload 'dired-rename-file "dired-aux" nil nil nil)
 (autoload 'display-call-tree "bytecomp" nil nil nil)
+(autoload 'display-table-slot "disp-table" nil nil nil)
 (autoload 'display-warning "warnings" nil nil nil)
 (autoload 'easy-menu-create-menu "easymenu" nil nil nil)
+(autoload 'edebug-basic-spec "edebug" nil nil nil)
 (autoload 'ediff-patch-file "ediff" nil nil nil)
 (autoload 'edit-kbd-macro "edmacro" nil nil nil)
 (autoload 'extract-rectangle "rect" nil nil nil)
@@ -67,8 +86,10 @@
 (autoload 'help-with-tutorial "tutorial" nil nil nil)
 (autoload 'help-xref-button "help-mode" nil nil nil)
 (autoload 'hi-lock-face-buffer "hi-lock" nil nil nil)
+(autoload 'html-mode "sgml-mode" nil nil nil)
 (autoload 'image-type-available-p "image" nil nil nil)
 (autoload 'info "info" nil nil nil)
+(autoload 'info-complete-symbol "info-look" nil nil nil)
 (autoload 'info-emacs-manual "info" nil nil nil)
 (autoload 'insert-image "image" nil nil nil)
 (autoload 'insert-rectangle "rect" nil nil nil)
@@ -83,21 +104,27 @@
 (autoload 'multi-isearch-buffers-regexp "misearch" nil nil nil)
 (autoload 'multi-isearch-files "misearch" nil nil nil)
 (autoload 'multi-isearch-files-regexp "misearch" nil nil nil)
+(autoload 'nxml-mode "nxml-mode" nil nil nil)
 (autoload 'open-network-stream "network-stream" nil nil nil)
 (autoload 'package-initialize "package" nil nil nil)
 (autoload 'parse-time-string "parse-time" nil nil nil)
 (autoload 'pp "pp" nil nil nil)
 (autoload 'pp-buffer "pp" nil nil nil)
+(autoload 'print-buffer "lpr" nil nil nil)
+(autoload 'quail-defrule-internal "quail" nil nil nil)
 (autoload 'read-kbd-macro "edmacro" nil nil nil)
 (autoload 'regexp-opt "regexp-opt" nil nil nil)
 (autoload 'rx "rx" nil nil t)
 (autoload 'seconds-to-string "time-date" nil nil nil)
 (autoload 'seconds-to-time "time-date" nil nil nil)
+(autoload 'server-save-buffers-kill-terminal "server" nil nil nil)
 (autoload 'server-start "server" nil nil nil)
 (autoload 'set-nested-alist "mule-util" nil nil nil)
+(autoload 'skeleton-insert "skeleton" nil nil nil)
 (autoload 'smerge-mode "smerge-mode" nil nil nil)
 (autoload 'smerge-start-session "smerge-mode" nil nil nil)
 (autoload 'standard-display-8bit "disp-table" nil nil nil)
+(autoload 'standard-display-default "disp-table" nil nil nil)
 (autoload 'tags-query-replace "etags" nil nil nil)
 (autoload 'tags-search "etags" nil nil nil)
 (autoload 'text-scale-increase "face-remap" nil nil nil)
@@ -106,6 +133,12 @@
 (autoload 'timezone-make-date-arpa-standard "timezone" nil nil nil)
 (autoload 'tmm-menubar "tmm" nil nil nil)
 (autoload 'truncate-string-to-width "mule-util" nil nil nil)
+(autoload 'ucs-normalize-HFS-NFC-region "ucs-normalize" nil nil nil)
+(autoload 'ucs-normalize-HFS-NFD-region "ucs-normalize" nil nil nil)
+(autoload 'ucs-normalize-NFC-region "ucs-normalize" nil nil nil)
+(autoload 'ucs-normalize-NFD-region "ucs-normalize" nil nil nil)
+(autoload 'ucs-normalize-NFKC-region "ucs-normalize" nil nil nil)
+(autoload 'ucs-normalize-NFKD-region "ucs-normalize" nil nil nil)
 (autoload 'url-handler-mode "url-handlers" nil nil nil)
 (autoload 'variable-at-point "help-fns" nil nil nil)
 (autoload 'vc-register "vc" nil nil nil)
@@ -114,6 +147,7 @@
 (autoload 'view-buffer "view" nil nil nil)
 (autoload 'view-buffer-other-window "view" nil nil nil)
 (autoload 'view-file "view" nil nil nil)
+(autoload 'view-mode "view" nil nil nil)
 (autoload 'view-mode-enter "view" nil nil nil)
 (autoload 'visit-tags-table "etags" nil nil nil)
 (autoload 'warn "warnings" nil nil nil)
-- 
2.7.4

bug#25360: File mode specification errors during building

[Eli Zaretskii]

> From: phillip.lord@russet.org.uk (Phillip Lord)
> Cc: rgm@gnu.org,  25360@debbugs.gnu.org
> Date: Fri, 13 Jan 2017 14:08:55 +0000
> 
> So, the cause of the problem is this. bootstrap-emacs (nor emacs) does
> not actually load loaddefs.el; it's just dumped with the binary. I was
> recording autoloads until loaddefs.el was generated, but actually, I
> need to record them till loaddefs.el is used, that is when emacs is
> dumped.
> 
> The attached patch seems to address. Can you confirm, then I will apply
> to master.

Looks OK, but I guess the comment before the changed code needs to be
updated as well?

Thanks.

bug#25360: File mode specification errors during building

[Phillip Lord]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: phillip.lord@russet.org.uk (Phillip Lord)
>> Cc: rgm@gnu.org,  25360@debbugs.gnu.org
>> Date: Fri, 13 Jan 2017 14:08:55 +0000
>> 
>> So, the cause of the problem is this. bootstrap-emacs (nor emacs) does
>> not actually load loaddefs.el; it's just dumped with the binary. I was
>> recording autoloads until loaddefs.el was generated, but actually, I
>> need to record them till loaddefs.el is used, that is when emacs is
>> dumped.
>> 
>> The attached patch seems to address. Can you confirm, then I will apply
>> to master.
>
> Looks OK, but I guess the comment before the changed code needs to be
> updated as well?

Good catch!

I've pushed to master. We can close this if the next hydra build works
as expected.

Phil

bug#25360: File mode specification errors during building

[Glenn Morris]

Phillip Lord wrote:

> I've pushed to master. We can close this if the next hydra build works
> as expected.

There's nothing special about hydra builds. I quote them only because
they provide a convenient way to show a full, clean build log.

The image-type-auto-detected-p issue persists, eg
http://hydra.nixos.org/build/46502443/log/raw

Note this is a without-x build (image-type-auto-detected-p used to be
autoloaded in all builds).

bug#25360: File mode specification errors during building

[Phillip Lord]

Glenn Morris <rgm@gnu.org> writes:

> Phillip Lord wrote:
>
>> I've pushed to master. We can close this if the next hydra build works
>> as expected.
>
> There's nothing special about hydra builds. I quote them only because
> they provide a convenient way to show a full, clean build log.
>
> The image-type-auto-detected-p issue persists, eg
> http://hydra.nixos.org/build/46502443/log/raw
>
> Note this is a without-x build (image-type-auto-detected-p used to be
> autoloaded in all builds).

Unfortunately, I cannot reproduce this on my own machine. Exactly how is
the hydra build configured? I've tried --without-x, as a guess, but I
get no error.

Possible solutions:

 1) Ignore it
 2) Move the leim-list dependency from "emacs" to "all". The leim files
    will now get built by emacs not bootstrap-emacs.
 3) Suppress the error in some other way, probably by sticking
    "input-type-auto-detected-p" into ldefs-boot-manual.el

I quite like 2, but it will break two autoloads
(hangul-input-method-activate and ucs-input-activate). I do not know why
these two methods need autoloading while none of the other files in the
leim directory have autoloads.

I'm struggling with 3) because I can't reproduce.


Phil

bug#25360: File mode specification errors during building

[npostavs]

phillip.lord@russet.org.uk (Phillip Lord) writes:

> Glenn Morris <rgm@gnu.org> writes:
>
>> Phillip Lord wrote:
>>
>>> I've pushed to master. We can close this if the next hydra build works
>>> as expected.
>>
>> There's nothing special about hydra builds. I quote them only because
>> they provide a convenient way to show a full, clean build log.
>>
>> The image-type-auto-detected-p issue persists, eg
>> http://hydra.nixos.org/build/46502443/log/raw
>>
>> Note this is a without-x build (image-type-auto-detected-p used to be
>> autoloaded in all builds).
>
> Unfortunately, I cannot reproduce this on my own machine. Exactly how is
> the hydra build configured? I've tried --without-x, as a guess, but I
> get no error.

I can reproduce here.  Did you just run 'make bootstrap' after
reconfiguring?  I think you need 'make extraclean', because the part
that triggers this is not cleaned by bootstrap.

Using insert-file-contents instead of file-file-noselect seems to fix it
for me:

--- i/lisp/international/titdic-cnv.el
+++ w/lisp/international/titdic-cnv.el
@@ -1167,11 +1167,11 @@ miscdic-convert
 		    (if (eq coding 'iso-2022-cn-ext) "Chinese-CNS"
 		      "Chinese-GB"))
 		  "\" \"" title "\" t\n")
-	  (let* ((coding-system-for-read
-		  (coding-system-change-eol-conversion coding 'unix))
-		 (dicbuf (find-file-noselect filename)))
-	    (funcall converter dicbuf name title)
-	    (kill-buffer dicbuf))
+          (let ((coding-system-for-read
+                 (coding-system-change-eol-conversion coding 'unix)))
+            (with-temp-buffer
+              (insert-file-contents filename)
+              (funcall converter (current-buffer) name title)))
 	  (insert ";; Local Variables:\n"
 		  ";; version-control: never\n"
 		  ";; no-update-autoloads: t\n"

bug#25360: File mode specification errors during building

[Phillip Lord]

npostavs@users.sourceforge.net writes:
>>> There's nothing special about hydra builds. I quote them only because
>>> they provide a convenient way to show a full, clean build log.
>>>
>>> The image-type-auto-detected-p issue persists, eg
>>> http://hydra.nixos.org/build/46502443/log/raw
>>>
>>> Note this is a without-x build (image-type-auto-detected-p used to be
>>> autoloaded in all builds).
>>
>> Unfortunately, I cannot reproduce this on my own machine. Exactly how is
>> the hydra build configured? I've tried --without-x, as a guess, but I
>> get no error.
>
> I can reproduce here.  Did you just run 'make bootstrap' after
> reconfiguring?  I think you need 'make extraclean', because the part
> that triggers this is not cleaned by bootstrap.

To my understanding, make bootstrap is at least as clean as extraclean.


> Using insert-file-contents instead of file-file-noselect seems to fix it
> for me:
>
> --- i/lisp/international/titdic-cnv.el
> +++ w/lisp/international/titdic-cnv.el
> @@ -1167,11 +1167,11 @@ miscdic-convert
>  		    (if (eq coding 'iso-2022-cn-ext) "Chinese-CNS"
>  		      "Chinese-GB"))
>  		  "\" \"" title "\" t\n")
> -	  (let* ((coding-system-for-read
> -		  (coding-system-change-eol-conversion coding 'unix))
> -		 (dicbuf (find-file-noselect filename)))
> -	    (funcall converter dicbuf name title)
> -	    (kill-buffer dicbuf))
> +          (let ((coding-system-for-read
> +                 (coding-system-change-eol-conversion coding 'unix)))
> +            (with-temp-buffer
> +              (insert-file-contents filename)
> +              (funcall converter (current-buffer) name title)))
>  	  (insert ";; Local Variables:\n"
>  		  ";; version-control: never\n"
>  		  ";; no-update-autoloads: t\n"

Confused. What thing are you reproducing? The
"image-type-auto-detected-p" or File mode specification error:
(void-function html-mode) error?

The "find-file-noselect" call is responsible for the "html-mode" error,
but that should have been fixed with this commit.

commit 72c668a9042ac6475eadedfee5c87fb1e6b2d753
Author: Phillip Lord <phillip.lord@russet.org.uk>
Date:   Fri Jan 13 13:57:51 2017 +0000

But this commit also stops the image-type-auto-detected-p errors for me.

Phil

bug#25360: File mode specification errors during building

[npostavs]

phillip.lord@russet.org.uk (Phillip Lord) writes:

> npostavs@users.sourceforge.net writes:
>>>> There's nothing special about hydra builds. I quote them only because
>>>> they provide a convenient way to show a full, clean build log.
>>>>
>>>> The image-type-auto-detected-p issue persists, eg
>>>> http://hydra.nixos.org/build/46502443/log/raw
>>>>
>>>> Note this is a without-x build (image-type-auto-detected-p used to be
>>>> autoloaded in all builds).
>>>
>>> Unfortunately, I cannot reproduce this on my own machine. Exactly how is
>>> the hydra build configured? I've tried --without-x, as a guess, but I
>>> get no error.
>>
>> I can reproduce here.  Did you just run 'make bootstrap' after
>> reconfiguring?  I think you need 'make extraclean', because the part
>> that triggers this is not cleaned by bootstrap.
>
> To my understanding, make bootstrap is at least as clean as extraclean.

I think extraclean does some more things, but actually it might not be
relevant in this case.

>
>
>> Using insert-file-contents instead of file-file-noselect seems to fix it
>> for me:
>>
>> --- i/lisp/international/titdic-cnv.el
>> +++ w/lisp/international/titdic-cnv.el
>> @@ -1167,11 +1167,11 @@ miscdic-convert
>>  		    (if (eq coding 'iso-2022-cn-ext) "Chinese-CNS"
>>  		      "Chinese-GB"))
>>  		  "\" \"" title "\" t\n")
>> -	  (let* ((coding-system-for-read
>> -		  (coding-system-change-eol-conversion coding 'unix))
>> -		 (dicbuf (find-file-noselect filename)))
>> -	    (funcall converter dicbuf name title)
>> -	    (kill-buffer dicbuf))
>> +          (let ((coding-system-for-read
>> +                 (coding-system-change-eol-conversion coding 'unix)))
>> +            (with-temp-buffer
>> +              (insert-file-contents filename)
>> +              (funcall converter (current-buffer) name title)))
>>  	  (insert ";; Local Variables:\n"
>>  		  ";; version-control: never\n"
>>  		  ";; no-update-autoloads: t\n"
>
> Confused. What thing are you reproducing? The
> "image-type-auto-detected-p" or File mode specification error:
> (void-function html-mode) error?

Here is the command I found was problematic during the build (I added
the rm -f part, otherwise the file generation is skipped).

cd /home/npostavs/src/emacs/emacs-bootstrapping/leim && rm -f ../lisp/leim/quail/tsang-b5.el ../lisp/leim/quail/quick-b5.el && EMACSLOADPATH= '../src/bootstrap-emacs' --batch --no-site-file --no-site-lisp -l titdic-cnv.el -f batch-miscdic-convert -dir ./../lisp/leim/quail MISC-DIC/cangjie-table.b5
Converting cangjie-table.b5 to tsang-b5.el...
File mode specification error: (wrong-type-argument stringp image-type-auto-detected-p)
Converting cangjie-table.b5 to tsang-b5.el...done
Converting cangjie-table.b5 to quick-b5.el...
File mode specification error: (wrong-type-argument stringp image-type-auto-detected-p)
Converting cangjie-table.b5 to quick-b5.el...done

The patch I posted fixes these errors.


> The "find-file-noselect" call is responsible for the "html-mode" error,
> but that should have been fixed with this commit.
>
> commit 72c668a9042ac6475eadedfee5c87fb1e6b2d753
> Author: Phillip Lord <phillip.lord@russet.org.uk>
> Date:   Fri Jan 13 13:57:51 2017 +0000
>
> But this commit also stops the image-type-auto-detected-p errors for me.

Does the following print something non-nil for you?  For me it's nil
when I configure --without-x, and a byte code function value otherwise.

./bootstrap-emacs -batch -Q --eval '(message "%S" (symbol-function (quote image-type-auto-detected-p)))'

bug#25360: File mode specification errors during building

[Glenn Morris]

Phillip Lord wrote:

> commit 72c668a9042ac6475eadedfee5c87fb1e6b2d753
> Author: Phillip Lord <phillip.lord@russet.org.uk>
> Date:   Fri Jan 13 13:57:51 2017 +0000
>
> But this commit also stops the image-type-auto-detected-p errors for me.

How can it, when it does not change anything related to
image-type-auto-detected-p?

bug#25360: File mode specification errors during building

[Phillip Lord]

Glenn Morris <rgm@gnu.org> writes:

> Phillip Lord wrote:
>
>> commit 72c668a9042ac6475eadedfee5c87fb1e6b2d753
>> Author: Phillip Lord <phillip.lord@russet.org.uk>
>> Date:   Fri Jan 13 13:57:51 2017 +0000
>>
>> But this commit also stops the image-type-auto-detected-p errors for me.
>
> How can it, when it does not change anything related to
> image-type-auto-detected-p?


Weird isn't it? Still, it's true.

I don't know exactly what is happening but, I think, the root part of
the problem comes from here in loadup.el.

(if (fboundp 'x-create-frame)
    (progn
      (load "fringe")
      ;; Needed by `imagemagick-register-types'
      (load "emacs-lisp/regexp-opt")
      (load "image")
      (load "international/fontset")
      (load "dnd")
      (load "tool-bar")))


What this means, AFAICT, is that in a --with-x build, x-create-frame
will exist, so image.el will be loaded and dumped. But, in a -without-x
build it will not be, so it must, therefore, be autoloaded if it used.

Phil

bug#25360: File mode specification errors during building

[Glenn Morris]

npostavs@users.sourceforge.net wrote:

> Using insert-file-contents instead of file-file-noselect seems to fix it
> for me:

That sounds like a good change anyway. (I forget, does this respect
"coding:"? Does this even matter in this case?)

(But it still seems like a potential problem to me if functions that used to
be available early in the build no longer are.)

bug#25360: File mode specification errors during building

[Phillip Lord]

[-- Attachment #1: Type: text/plain, Size: 1372 bytes --]

Glenn Morris <rgm@gnu.org> writes:

> npostavs@users.sourceforge.net wrote:
>
>> Using insert-file-contents instead of file-file-noselect seems to fix it
>> for me:
>
> That sounds like a good change anyway. (I forget, does this respect
> "coding:"? Does this even matter in this case?)

I think, no, it doesn't respect coding, but that it doesn't matter.

>
> (But it still seems like a potential problem to me if functions that used to
> be available early in the build no longer are.)

It shouldn't be, if they are not being called.

Anyway, the cause of the error in this case is this line in files.el.

	 (assoc-default nil magic-fallback-mode-alist
		(lambda (re _dummy)
		  (if (functionp re)
		      (funcall re)
		    (looking-at re)))))))

This is called when loading cangjie-table.b5 with
'image-type-auto-detected-p as the first parameter. Normally, the
condition returns "t", but in this case it will return f, then
the looking-at form fails with the error given. I don't understand which
this process does not happen during the build which generates
ldefs-boot-auto.el.

The bigger problem is that the bootstrap-emacs executable has different
functionality compiled in, even when it does not require it to perform
the task of bootstrapping. This might invalidate my approach, but I am
not sure yet.

The follow patch addresses, the problem at hand, though.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Fix-file-mode-errors-in-bootstrap.patch --]
[-- Type: text/x-diff, Size: 3582 bytes --]

From bed019eb18fbafffd397eaac7962e30428fe1e04 Mon Sep 17 00:00:00 2001
From: Phillip Lord <phillip.lord@russet.org.uk>
Date: Mon, 16 Jan 2017 23:09:06 +0000
Subject: [PATCH] Fix file-mode errors in bootstrap.

* lisp/files.el (set-auto-mode): Do not call `looking-at' on a symbol.
* lisp/international/titdic-cnv.el (miscdic-convert): Use
  `insert-file-contents' rather than `find-file-noselect'.
---
 lisp/files.el                    | 28 ++++++++++++++++++++++------
 lisp/international/titdic-cnv.el |  8 ++++----
 2 files changed, 26 insertions(+), 10 deletions(-)

diff --git a/lisp/files.el b/lisp/files.el
index b57e35b..e7b15a8 100644
--- a/lisp/files.el
+++ b/lisp/files.el
@@ -2911,9 +2911,18 @@ set-auto-mode
 						(+ (point-min) magic-mode-regexp-match-limit)))
 			 (assoc-default nil magic-mode-alist
 					(lambda (re _dummy)
-					  (if (functionp re)
-					      (funcall re)
-					    (looking-at re)))))))
+                                          (cond
+                                           ;; Bug#25360 We might have a
+                                           ;; symbol which is not a
+                                           ;; function during bootstrap.
+                                           ((and
+                                              (symbolp re)
+                                              (functionp re))
+                                            (funcall re))
+                                           ((stringp re)
+					    (looking-at re)))
+                                          (t (error "Problem with `magic-mode-alist'."))
+                                           )))))
 	  (set-auto-mode-0 done keep-mode-if-same)))
     ;; Next compare the filename against the entries in auto-mode-alist.
     (unless done
@@ -2966,9 +2975,16 @@ set-auto-mode
 						(+ (point-min) magic-mode-regexp-match-limit)))
 			 (assoc-default nil magic-fallback-mode-alist
 					(lambda (re _dummy)
-					  (if (functionp re)
-					      (funcall re)
-					    (looking-at re)))))))
+                                          ;; Bug#25360 We might have a
+                                          ;; symbol which is not a
+                                          ;; function during bootstrap.
+					  (cond
+                                           ((functionp re)
+                                            (funcall re))
+                                           ((stringp re)
+                                            (looking-at re))
+                                           (t (error "Problem with `magic-fallback-mode-alist'."))
+                                           ))))))
 	  (set-auto-mode-0 done keep-mode-if-same)))
     (unless done
       (set-buffer-major-mode (current-buffer)))))
diff --git a/lisp/international/titdic-cnv.el b/lisp/international/titdic-cnv.el
index 6f65d49..6986b3f 100644
--- a/lisp/international/titdic-cnv.el
+++ b/lisp/international/titdic-cnv.el
@@ -1168,10 +1168,10 @@ miscdic-convert
 		      "Chinese-GB"))
 		  "\" \"" title "\" t\n")
 	  (let* ((coding-system-for-read
-		  (coding-system-change-eol-conversion coding 'unix))
-		 (dicbuf (find-file-noselect filename)))
-	    (funcall converter dicbuf name title)
-	    (kill-buffer dicbuf))
+		  (coding-system-change-eol-conversion coding 'unix)))
+            (with-temp-buffer
+              (insert-file-contents filename)
+              (funcall converter (current-buffer) name title)))
 	  (insert ";; Local Variables:\n"
 		  ";; version-control: never\n"
 		  ";; no-update-autoloads: t\n"
-- 
2.9.3

bug#25360: File mode specification errors during building

[npostavs]

phillip.lord@russet.org.uk (Phillip Lord) writes:

> Glenn Morris <rgm@gnu.org> writes:
>
>> npostavs@users.sourceforge.net wrote:
>>
>>> Using insert-file-contents instead of file-file-noselect seems to fix it
>>> for me:
>>
>> That sounds like a good change anyway. (I forget, does this respect
>> "coding:"? Does this even matter in this case?)
>
> I think, no, it doesn't respect coding, but that it doesn't matter.

I believe it does respect "coding:" in general (at least it looks like
there are some comments to that effect in Finsert_file_contents), but
indeed it doesn't matter in this case because we're let-binding
`coding-system-for-read' which overrides that.

>
>>
>> (But it still seems like a potential problem to me if functions that used to
>> be available early in the build no longer are.)
>
> It shouldn't be, if they are not being called.
>
> Anyway, the cause of the error in this case is this line in files.el.
>
> 	 (assoc-default nil magic-fallback-mode-alist
> 		(lambda (re _dummy)
> 		  (if (functionp re)
> 		      (funcall re)
> 		    (looking-at re)))))))
>
> This is called when loading cangjie-table.b5 with
> 'image-type-auto-detected-p as the first parameter. Normally, the
> condition returns "t", but in this case it will return f, then
> the looking-at form fails with the error given. I don't understand which
> this process does not happen during the build which generates
> ldefs-boot-auto.el.

Would it help to use a --without-x build to generate ldefs-boot-auto.el?

> +                                          (t (error "Problem with `magic-mode-alist'."))

I'm not sure this improves the error message ("Problem" seems a bit
vague).  Before:

    Converting cangjie-table.b5 to tsang-b5.el...
    File mode specification error: (wrong-type-argument stringp image-type-auto-detected-p)
    Converting cangjie-table.b5 to tsang-b5.el...done

After (applying only the lisp/files.el part of your patch):

    Converting cangjie-table.b5 to tsang-b5.el...
    File mode specification error: (error Problem with ¥magic-fallback-mode-alist¦.)
    Converting cangjie-table.b5 to tsang-b5.el...done

Perhaps add in the problematic element to the message?  Something along
the lines of (error "Bad `magic-mode-alist' element: %S" re).

bug#25360: File mode specification errors during building

[Phillip Lord]

npostavs@users.sourceforge.net writes:

> phillip.lord@russet.org.uk (Phillip Lord) writes:
>
>> Glenn Morris <rgm@gnu.org> writes:
>>
>>> npostavs@users.sourceforge.net wrote:
>>>
>>>> Using insert-file-contents instead of file-file-noselect seems to fix it
>>>> for me:
>>>
>>> That sounds like a good change anyway. (I forget, does this respect
>>> "coding:"? Does this even matter in this case?)
>>
>> I think, no, it doesn't respect coding, but that it doesn't matter.
>
> I believe it does respect "coding:" in general (at least it looks like
> there are some comments to that effect in Finsert_file_contents), but
> indeed it doesn't matter in this case because we're let-binding
> `coding-system-for-read' which overrides that.

I have checked this now with a full bootstrap and it appears not to
work, unfortunately -- the .el files that are generated are broken (this
isn't obvious until they fail to byte-compile -- you have let the build
complete).

So, find-file-noselect is doing something that insert-file-contents is not.


>>> (But it still seems like a potential problem to me if functions that used to
>>> be available early in the build no longer are.)
>>
>> It shouldn't be, if they are not being called.
>>
>> Anyway, the cause of the error in this case is this line in files.el.
>>
>> 	 (assoc-default nil magic-fallback-mode-alist
>> 		(lambda (re _dummy)
>> 		  (if (functionp re)
>> 		      (funcall re)
>> 		    (looking-at re)))))))
>>
>> This is called when loading cangjie-table.b5 with
>> 'image-type-auto-detected-p as the first parameter. Normally, the
>> condition returns "t", but in this case it will return f, then
>> the looking-at form fails with the error given. I don't understand which
>> this process does not happen during the build which generates
>> ldefs-boot-auto.el.
>
> Would it help to use a --without-x build to generate ldefs-boot-auto.el?


I'm a little loath to do this, because if a developer wants to run this,
it will reconfigure their build.

I've tried instead disabling all the "optional" statements in loadup.el
when building the bootstrap binary. bootstrap-emacs doesn't use images
or toolbars anyway. Will work up a patch tonight.




>
>> + (t (error "Problem with `magic-mode-alist'."))
>
> I'm not sure this improves the error message ("Problem" seems a bit
> vague).  Before:
>
>     Converting cangjie-table.b5 to tsang-b5.el...
>     File mode specification error: (wrong-type-argument stringp
> image-type-auto-detected-p)
>     Converting cangjie-table.b5 to tsang-b5.el...done
>
> After (applying only the lisp/files.el part of your patch):
>
>     Converting cangjie-table.b5 to tsang-b5.el...
>     File mode specification error: (error Problem with
> ¥magic-fallback-mode-alist¦.)
>     Converting cangjie-table.b5 to tsang-b5.el...done
>
> Perhaps add in the problematic element to the message?  Something along
> the lines of (error "Bad `magic-mode-alist' element: %S" re).

Yes, realised I can't use backquotes in error messages.

Your change is reasonable. In practice, I think, only developers are
likely to see this, and either of these messages is grepable, unlike the
wrong-type-argument message.

Phil

bug#25360: File mode specification errors during building

[Eli Zaretskii]

> From: phillip.lord@russet.org.uk (Phillip Lord)
> Cc: Glenn Morris <rgm@gnu.org>,  25360@debbugs.gnu.org,  Eli Zaretskii <eliz@gnu.org>
> Date: Thu, 19 Jan 2017 10:45:51 +0000
> 
> > I believe it does respect "coding:" in general (at least it looks like
> > there are some comments to that effect in Finsert_file_contents), but
> > indeed it doesn't matter in this case because we're let-binding
> > `coding-system-for-read' which overrides that.
> 
> I have checked this now with a full bootstrap and it appears not to
> work, unfortunately -- the .el files that are generated are broken (this
> isn't obvious until they fail to byte-compile -- you have let the build
> complete).

Every file is broken, or just some of them?  If the latter, can you
name some of them, or try to figure out what they have in common?

bug#25360: File mode specification errors during building

[Noam Postavsky]

[-- Attachment #1: Type: text/plain, Size: 1142 bytes --]

On Thu, Jan 19, 2017 at 11:05 AM, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: phillip.lord@russet.org.uk (Phillip Lord)
>> Cc: Glenn Morris <rgm@gnu.org>,  25360@debbugs.gnu.org,  Eli Zaretskii <eliz@gnu.org>
>> Date: Thu, 19 Jan 2017 10:45:51 +0000
>>
>> > I believe it does respect "coding:" in general (at least it looks like
>> > there are some comments to that effect in Finsert_file_contents), but
>> > indeed it doesn't matter in this case because we're let-binding
>> > `coding-system-for-read' which overrides that.
>>
>> I have checked this now with a full bootstrap and it appears not to
>> work, unfortunately -- the .el files that are generated are broken (this
>> isn't obvious until they fail to byte-compile -- you have let the build
>> complete).
>
> Every file is broken, or just some of them?  If the latter, can you
> name some of them, or try to figure out what they have in common?

The problem is that (with-temp-buffer) changes the current buffer,
which the converter function expected to be set to the output file
(sorry for not checking this properly before). Attached is a
non-broken version of my previous patch.

[-- Attachment #2: miscdic-convert-v2.diff --]
[-- Type: text/plain, Size: 1059 bytes --]

diff --git i/lisp/international/titdic-cnv.el w/lisp/international/titdic-cnv.elindex 6f65d49..130bc74 100644
--- i/lisp/international/titdic-cnv.el
+++ w/lisp/international/titdic-cnv.el
@@ -1167,11 +1167,14 @@ miscdic-convert
 		    (if (eq coding 'iso-2022-cn-ext) "Chinese-CNS"
 		      "Chinese-GB"))
 		  "\" \"" title "\" t\n")
-	  (let* ((coding-system-for-read
-		  (coding-system-change-eol-conversion coding 'unix))
-		 (dicbuf (find-file-noselect filename)))
-	    (funcall converter dicbuf name title)
-	    (kill-buffer dicbuf))
+          (let ((coding-system-for-read
+                 (coding-system-change-eol-conversion coding 'unix))
+                (dstbuf (current-buffer)))
+            (with-temp-buffer
+              (insert-file-contents filename)
+              (let ((dicbuf (current-buffer)))
+                (with-current-buffer dstbuf
+                  (funcall converter dicbuf name title)))))
 	  (insert ";; Local Variables:\n"
 		  ";; version-control: never\n"
 		  ";; no-update-autoloads: t\n"

bug#25360: File mode specification errors during building

[Phillip Lord]

Noam Postavsky <npostavs@users.sourceforge.net> writes:

> On Thu, Jan 19, 2017 at 11:05 AM, Eli Zaretskii <eliz@gnu.org> wrote:
>>> From: phillip.lord@russet.org.uk (Phillip Lord)
>>> Cc: Glenn Morris <rgm@gnu.org>,  25360@debbugs.gnu.org,  Eli Zaretskii <eliz@gnu.org>
>>> Date: Thu, 19 Jan 2017 10:45:51 +0000
>>>
>>> > I believe it does respect "coding:" in general (at least it looks like
>>> > there are some comments to that effect in Finsert_file_contents), but
>>> > indeed it doesn't matter in this case because we're let-binding
>>> > `coding-system-for-read' which overrides that.
>>>
>>> I have checked this now with a full bootstrap and it appears not to
>>> work, unfortunately -- the .el files that are generated are broken (this
>>> isn't obvious until they fail to byte-compile -- you have let the build
>>> complete).
>>
>> Every file is broken, or just some of them?  If the latter, can you
>> name some of them, or try to figure out what they have in common?
>
> The problem is that (with-temp-buffer) changes the current buffer,
> which the converter function expected to be set to the output file
> (sorry for not checking this properly before). Attached is a
> non-broken version of my previous patch.

Ah, yes, you are right. Nothing to do with insert-file-contents at
all. Entirely obvious in hindsight, but then isn't it always.

Phil

bug#25360: File mode specification errors during building

[Phillip Lord]

I've pushed three commits to fix/bootstrap-build-minimize which should
solve all of the problems. titdic-cnv no longer uses find-file (which
causes the errors), the functions in bootstrap-emacs have been minimized
so that it should be indentical on all platforms (which, ironically,
increases the size of ldefs-boot-auto), and I've added error-detection
to files.el.

I need to build this on windows, and if anyone else has the time to do a
bootstrap build that would be good.


Phil

bug#25360: File mode specification errors during building

[Phillip Lord]

phillip.lord@russet.org.uk (Phillip Lord) writes:

> I've pushed three commits to fix/bootstrap-build-minimize which should
> solve all of the problems. titdic-cnv no longer uses find-file (which
> causes the errors), the functions in bootstrap-emacs have been minimized
> so that it should be indentical on all platforms (which, ironically,
> increases the size of ldefs-boot-auto), and I've added error-detection
> to files.el.
>
> I need to build this on windows, and if anyone else has the time to do a
> bootstrap build that would be good.

The final version of this is on fix/bootstrap-build-minimize-squash,
tested on windows and gnu/linux. Comments welcome or I'll push this to
master tomorrow.

Phil

bug#25360: File mode specification errors during building

[npostavs]

phillip.lord@russet.org.uk (Phillip Lord) writes:

> phillip.lord@russet.org.uk (Phillip Lord) writes:
>
>> I've pushed three commits to fix/bootstrap-build-minimize which should
>> solve all of the problems. titdic-cnv no longer uses find-file (which
>> causes the errors), the functions in bootstrap-emacs have been minimized
>> so that it should be indentical on all platforms (which, ironically,
>> increases the size of ldefs-boot-auto), and I've added error-detection
>> to files.el.
>>
>> I need to build this on windows, and if anyone else has the time to do a
>> bootstrap build that would be good.
>
> The final version of this is on fix/bootstrap-build-minimize-squash,
> tested on windows and gnu/linux. Comments welcome or I'll push this to
> master tomorrow.

Just a few minor formatting things:

> Remove conditional includes from bootstrap
> 
>  * lisp/loadup.el: No longer load optional includes during bootstrap
>    dumping.
>  * lisp/ldefs-boot-auto.el: Regenerate.
>  * lisp/ldefs-boot-manual.el: Add two autoloads.
> 
> Previously, bootstrap-emacs includes optional functionality, depending
> on the platform which is not needed for bootstrap function. As a result,
> bootstrap-emacs contains different functions in different
> circumstances. If ldefs-boot-auto.el is generated, then loaded functions
> will not be added to ldefs-boot-auto.el, although they may be required
> during some builds. With this change, bootstrap-emacs should always
> behave the same way and, therefore, require the same autoloads.

Sentences should end in double space, and summary should before
ChangeLog entry.

> +;; These two autoloads are needed for files.el. They are only used on
> +;; their respective platforms so do not get added to
> +;; ldefs-boot-auto.el when it is generated on a different platform.
> +(autoload 'dos-convert-standard-filename "dos-fns.el" nil nil nil)
> +(autoload ' w32-convert-standard-filename "w32-fns.el" nil nil nil)

Extra space here.

In the last commit, "Add error handling to magic-mode-alist":

> +                                           (t
> +                                            (error
> +                                             "Problem in magic-mode-alist with element %s" re))
> +                                           ))))))

The close paren placement looks odd.

bug#25360: File mode specification errors during building

[Phillip Lord]

npostavs@users.sourceforge.net writes:

> phillip.lord@russet.org.uk (Phillip Lord) writes:
>
>> phillip.lord@russet.org.uk (Phillip Lord) writes:
>>
>> The final version of this is on fix/bootstrap-build-minimize-squash,
>> tested on windows and gnu/linux. Comments welcome or I'll push this to
>> master tomorrow.
>
> Just a few minor formatting things:
>
>> Remove conditional includes from bootstrap
>> 
>>  * lisp/loadup.el: No longer load optional includes during bootstrap
>>    dumping.
>>  * lisp/ldefs-boot-auto.el: Regenerate.
>>  * lisp/ldefs-boot-manual.el: Add two autoloads.
>> 
>> Previously, bootstrap-emacs includes optional functionality, depending
>> on the platform which is not needed for bootstrap function. As a result,
>> bootstrap-emacs contains different functions in different
>> circumstances. If ldefs-boot-auto.el is generated, then loaded functions
>> will not be added to ldefs-boot-auto.el, although they may be required
>> during some builds. With this change, bootstrap-emacs should always
>> behave the same way and, therefore, require the same autoloads.
>
> Sentences should end in double space, and summary should before
> ChangeLog entry.

Double space I seem to always get wrong, and I need to check my
configuration to stop if removing them.

The summary before the changelog, I am a bit less sure on. I see both
forms in use.


>> +(autoload 'dos-convert-standard-filename "dos-fns.el" nil nil nil)
>> +(autoload ' w32-convert-standard-filename "w32-fns.el" nil nil nil)
>
> Extra space here.
>
> In the last commit, "Add error handling to magic-mode-alist":
>
>> +                                           (t
>> +                                            (error
>> + "Problem in magic-mode-alist with element %s" re))
>> +                                           ))))))
>
> The close paren placement looks odd.


Will address these!

Did you have chance to check this on a bootstrap build? Would be good to
know I am not the only one.

Phil

bug#25360: File mode specification errors during building

[npostavs]

phillip.lord@russet.org.uk (Phillip Lord) writes:
>>
>> Sentences should end in double space, and summary should before
>> ChangeLog entry.
>
> The summary before the changelog, I am a bit less sure on. I see both
> forms in use.

Well, CONTRIBUTE says this:

    - Explaining the rationale for a design choice is best done in
      comments in the source code.  However, sometimes it is useful to
      describe just the rationale for a change; that can be done in the
      commit message between the summary line and the file entries.

>
> Did you have chance to check this on a bootstrap build? Would be good to
> know I am not the only one.

I did have a successful bootstrap with your previous
fix/bootstrap-build-minimize branch on Arch GNU/Linux,
--with-x-toolkit=lucid.  Haven't built the new branch yet, though it
seems to have the same diffs.

bug#25360: File mode specification errors during building

[Eli Zaretskii]

> From: phillip.lord@russet.org.uk (Phillip Lord)
> Date: Mon, 23 Jan 2017 12:44:57 +0000
> Cc: 25360@debbugs.gnu.org
> 
> The final version of this is on fix/bootstrap-build-minimize-squash,
> tested on windows and gnu/linux. Comments welcome or I'll push this to
> master tomorrow.

Thanks, but please leave more time for people to comment; one day is
not enough.  At least one week, or, better, two, should be reserved
for that.

bug#25360: File mode specification errors during building

[Phillip Lord]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: phillip.lord@russet.org.uk (Phillip Lord)
>> Date: Mon, 23 Jan 2017 12:44:57 +0000
>> Cc: 25360@debbugs.gnu.org
>> 
>> The final version of this is on fix/bootstrap-build-minimize-squash,
>> tested on windows and gnu/linux. Comments welcome or I'll push this to
>> master tomorrow.
>
> Thanks, but please leave more time for people to comment; one day is
> not enough.  At least one week, or, better, two, should be reserved
> for that.

Okay, I will wait till next week, although Noam has already commented.

Phil

bug#25360: File mode specification errors during building

[Eli Zaretskii]

> From: phillip.lord@russet.org.uk (Phillip Lord)
> Cc: npostavs@users.sourceforge.net,  25360@debbugs.gnu.org
> Date: Tue, 24 Jan 2017 12:51:00 +0000
> 
> > Thanks, but please leave more time for people to comment; one day is
> > not enough.  At least one week, or, better, two, should be reserved
> > for that.
> 
> Okay, I will wait till next week, although Noam has already commented.

Yes, Noam is lightning-fast ;-)  Others might need more time.

Thanks.

bug#25360: File mode specification errors during building

[Phillip Lord]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: phillip.lord@russet.org.uk (Phillip Lord)
>> Cc: npostavs@users.sourceforge.net,  25360@debbugs.gnu.org
>> Date: Tue, 24 Jan 2017 12:51:00 +0000
>> 
>> > Thanks, but please leave more time for people to comment; one day is
>> > not enough.  At least one week, or, better, two, should be reserved
>> > for that.
>> 
>> Okay, I will wait till next week, although Noam has already commented.
>
> Yes, Noam is lightning-fast ;-)  Others might need more time.
>
> Thanks.


Are you happy for me to install this now?

Phil

bug#25360: File mode specification errors during building

[Eli Zaretskii]

> From: phillip.lord@russet.org.uk (Phillip Lord)
> Cc: npostavs@users.sourceforge.net,  25360@debbugs.gnu.org
> Date: Mon, 06 Feb 2017 10:31:12 +0000
> 
> Are you happy for me to install this now?

Sure, go ahead.

Thanks.

bug#25360: File mode specification errors during building

[Glenn Morris]

BTW, I think this loaddefs stuff is now giving a bootstrap failure
related to the "used to be in ldefs-boot" string-to-list.

http://hydra.nixos.org/build/48432485

bug#25360: File mode specification errors during building

[Glenn Morris]

Glenn Morris wrote:

> BTW, I think this loaddefs stuff is now giving a bootstrap failure
> related to the "used to be in ldefs-boot" string-to-list.
>
> http://hydra.nixos.org/build/48432485

Since it's impossible to predict which autoloaded functions are going to
be needed in future bootstraps, no longer having a comprehensive
ldefs-boot seems to mean more cryptic failures of this sort are inevitable.
(Previously, this kind of thing could only happen if a new autoload
was added that was needed for bootstrap, which was rare and hopefully
obvious.)

bug#25360: File mode specification errors during building

[Glenn Morris]

Actually, now that I think a bit more, maybe this would have always
failed, with "attempt to autoload while preparing to dump".

bug#25360: File mode specification errors during building

[Phillip Lord]

Glenn Morris <rgm@gnu.org> writes:

> Glenn Morris wrote:
>
>> BTW, I think this loaddefs stuff is now giving a bootstrap failure
>> related to the "used to be in ldefs-boot" string-to-list.
>>
>> http://hydra.nixos.org/build/48432485
>
> Since it's impossible to predict which autoloaded functions are going to
> be needed in future bootstraps, no longer having a comprehensive
> ldefs-boot seems to mean more cryptic failures of this sort are inevitable.
> (Previously, this kind of thing could only happen if a new autoload
> was added that was needed for bootstrap, which was rare and hopefully
> obvious.)


Yes, you are correct; how often this would happen was the unknown
question.

I have checked this bug. I got it to reproduce after I realised that
you'd fixed it on master.

First, "make generate-ldefs-boot" does work correctly update
ldefs-boot-auto.el, actually by including "with-coding-priority". This
isn't as clean as I would hope because make generate-ldefs-boot won't
run from a clean checkout in these circumstances.

Second, I tracked down the cause of the problem -- in this case, it's

d8cca4d8c56a90ec9215d7bfb0b0edfa3a36ad4f

Specifically, it's here:

-(defcustom query-replace-from-to-separator
-  (propertize (if (char-displayable-p ?→) " → " " -> ")
-              'face 'minibuffer-prompt)
-  "String that separates FROM and TO in the history of replacement pairs."
-  ;; Avoids error when attempt to autoload char-displayable-p fails
-  ;; while preparing to dump, also stops customize-rogue listing this.
-  :initialize 'custom-initialize-delay
+(defcustom query-replace-from-to-separator " → "
+  "String that separates FROM and TO in the history of replacement pairs.
+When nil, the pair will not be added to the history (same behavior
+as in emacs 24.5)."
   :group 'matching
-  :type '(choice string (sexp :tag "Display specification"))
+  :type '(choice
+          (const :tag "Disabled" nil)
+          string)

Before this, the call to char-displayable-p would have forced loading of
mule-utils.el.

Unfortunately, as you say this is not predictable from the commit. You
have to know that char-displayable-p is autoloaded *and* that it is this
call to it which is forcing load of mule-utils (and also the associated
functions). It's also not possible to detect without a bootstrap build:
hydra gives this feedback pretty quickly, but obviously having trunk
broken for any time is not ideal.

Solutions:

1) Wait -- it's possible that this will be a rare occurence, and it
leaves us with a better understanding of how bootstrap works. My code is
mostly working as intended.

2) Revert and replace with a non-emacs mechanism for generated
ldefs-boot.el, which is likely to be awk. This would solve this issue
(as well as the "new autoload for bootstrap" problem). It might
introduce a problem because we'd have two different mechanisms for
making autoload files.

3) Revert and decide this is more effort than it's worth

4) Add "make ldefs-generate-boot" to the git commit hook.


I'm wavering between 1 and 2; for the latter I'll have to learn
awk. Number 4 wasn't a serious suggestion.

Phil

bug#25360: File mode specification errors during building

[Glenn Morris]

Another issue that I think is related to this:
Now there are lots of free variable (and unused lexical, due to missing
defvars) warnings during bootstrapping, about autoloaded variables. The
new ldefs stuff doesn't seem to handle variables at all?

Ref eg
http://hydra.nixos.org/build/48993155/log/raw

    In toplevel form:
    files.el:750:1:Warning: Unused lexical variable `tramp-mode'

    In directory-files-recursively:
    files.el:761:10:Warning: reference to free variable `tramp-mode'

    In find-alternate-file:
    files.el:1740:17:Warning: reference to free variable `dired-directory'
    files.el:1750:15:Warning: assignment to free variable `dired-directory'

etc

bug#25360: File mode specification errors during building

[Phillip Lord]

No, it doesn't, and I can see no obvious way of achieving this.

Perhaps it doesn't matter. These warnings will only occur during a full
bootstrap. So just switching the warnings off during bootstrap would
solve the problem. But this is also unsatisfying in other ways, and
probably more unsatisfying that just having ldefs-boot.el as before.

I have thought about the awk solution also. This seems to me to be
pretty hard, simply because the rules for doing autoloads are
complicated. Another possibility is to shrink loadup.el so that
bootstrap-emacs is more limited in functionality still.

At this point, though, I have to admit that I am leaning toward just
reverting.

Phil

Glenn Morris <rgm@gnu.org> writes:

> Another issue that I think is related to this:
> Now there are lots of free variable (and unused lexical, due to missing
> defvars) warnings during bootstrapping, about autoloaded variables. The
> new ldefs stuff doesn't seem to handle variables at all?
>
> Ref eg
> http://hydra.nixos.org/build/48993155/log/raw
>
>     In toplevel form:
>     files.el:750:1:Warning: Unused lexical variable `tramp-mode'
>
>     In directory-files-recursively:
>     files.el:761:10:Warning: reference to free variable `tramp-mode'
>
>     In find-alternate-file:
>     files.el:1740:17:Warning: reference to free variable `dired-directory'
>     files.el:1750:15:Warning: assignment to free variable `dired-directory'
>
> etc

bug#25360: File mode specification errors during building

[Glenn Morris]

Phillip Lord wrote:

> These warnings will only occur during a full bootstrap.

IMO that's the only way to do a proper build with informative warnings.

> So just switching the warnings off during bootstrap would solve the
> problem.

But then we can't spot when a "real" warning appears.

> But this is also unsatisfying in other ways, and probably more
> unsatisfying that just having ldefs-boot.el as before.

Agreed.

> I have thought about the awk solution also. This seems to me to be
> pretty hard, simply because the rules for doing autoloads are
> complicated.

Agreed.

> At this point, though, I have to admit that I am leaning toward just
> reverting.

I feel the same. I don't think the old system was causing any problems.

bug#25360: File mode specification errors during building

[Phillip Lord]

I've added changes to master which means that ldefs-boot-auto can be
regenerated without a bootstrap build. If the build breaks again, it can
be refixed rapidly now. I'll leave this for a while. If it breaks often,
I'll revert, but only time can tell.

Phil


phillip.lord@russet.org.uk (Phillip Lord) writes:

> No, it doesn't, and I can see no obvious way of achieving this.
>
> Perhaps it doesn't matter. These warnings will only occur during a full
> bootstrap. So just switching the warnings off during bootstrap would
> solve the problem. But this is also unsatisfying in other ways, and
> probably more unsatisfying that just having ldefs-boot.el as before.
>
> I have thought about the awk solution also. This seems to me to be
> pretty hard, simply because the rules for doing autoloads are
> complicated. Another possibility is to shrink loadup.el so that
> bootstrap-emacs is more limited in functionality still.
>
> At this point, though, I have to admit that I am leaning toward just
> reverting.
>
> Phil
>
> Glenn Morris <rgm@gnu.org> writes:
>
>> Another issue that I think is related to this:
>> Now there are lots of free variable (and unused lexical, due to missing
>> defvars) warnings during bootstrapping, about autoloaded variables. The
>> new ldefs stuff doesn't seem to handle variables at all?
>>
>> Ref eg
>> http://hydra.nixos.org/build/48993155/log/raw
>>
>>     In toplevel form:
>>     files.el:750:1:Warning: Unused lexical variable `tramp-mode'
>>
>>     In directory-files-recursively:
>>     files.el:761:10:Warning: reference to free variable `tramp-mode'
>>
>>     In find-alternate-file:
>>     files.el:1740:17:Warning: reference to free variable `dired-directory'
>>     files.el:1750:15:Warning: assignment to free variable `dired-directory'
>>
>> etc

bug#25360: File mode specification errors during building

[Eli Zaretskii]

> From: phillip.lord@russet.org.uk (Phillip Lord)
> Date: Wed, 01 Mar 2017 16:55:54 +0000
> Cc: npostavs@users.sourceforge.net, 25360@debbugs.gnu.org
> 
> 
> I've added changes to master which means that ldefs-boot-auto can be
> regenerated without a bootstrap build. If the build breaks again, it can
> be refixed rapidly now. I'll leave this for a while. If it breaks often,
> I'll revert, but only time can tell.

I see multiple error messages about w32-convert-standard-filename
during a normal build:

  Loading d:/gnu/git/emacs/trunk/lisp/leim/leim-list.el (source)...
  Finding pointers to doc strings...
  Finding pointers to doc strings...done
  Dumping under the name emacs
  Dump memory usage: Heap: 9953280  Large blocks(0/1): 0/786448
  Dumping from d:/gnu/git/emacs/trunk/src/temacs.exe
	    to d:/gnu/git/emacs/trunk/src/emacs.exe
  49515 pure bytes used
  mv -f emacs.exe bootstrap-emacs.exe
  make -C ../lisp compile-first EMACS="../src/bootstrap-emacs.exe"
  make[2]: Entering directory `/d/gnu/git/emacs/trunk/lisp'
    GEN      mh-e/mh-loaddefs.el
  Symbol's function definition is void: w32-convert-standard-filename
  Directories for loaddefs: . ./calc ./calendar ./cedet ./cedet/ede ./cedet/semantic ./cedet/semantic/analyze ./cedet/semantic/bovine ./cedet/semantic/decorate ./cedet/semantic/symref ./cedet/semantic/wisent ./cedet/srecode ./emacs-lisp ./emulation ./erc ./eshell ./gnus ./image ./international ./language ./leim ./leim/ja-dic ./leim/quail ./mail ./mh-e ./net ./nxml ./org ./play ./progmodes ./textmodes ./url ./vc
    GEN      loaddefs.el
  Symbol's function definition is void: w32-convert-standard-filename
  make[2]: Leaving directory `/d/gnu/git/emacs/trunk/lisp'
  make -C ../lisp autoloads EMACS="../src/bootstrap-emacs.exe"
  make -C ../admin/unidata all EMACS="../../src/bootstrap-emacs.exe"
  make -C ../leim leim-list.el EMACS="../src/bootstrap-emacs.exe"
  make[2]: Entering directory `/d/gnu/git/emacs/trunk/admin/unidata'
  make[2]: Entering directory `/d/gnu/git/emacs/trunk/leim'
  make[2]: Nothing to be done for `leim-list.el'.
  make[2]: Leaving directory `/d/gnu/git/emacs/trunk/leim'
  make[2]: Leaving directory `/d/gnu/git/emacs/trunk/admin/unidata'
  make[2]: Entering directory `/d/gnu/git/emacs/trunk/lisp'
  make[2]: Entering directory `/d/gnu/git/emacs/trunk/lisp'
    ELC      ../lisp/files.elc
    GEN      mh-e/mh-loaddefs.el
  make[2]: Entering directory `/d/gnu/git/emacs/trunk/lisp'
    ELC      ../lisp/international/mule.elc
  Symbol's function definition is void: w32-convert-standard-filename
  Symbol's function definition is void: w32-convert-standard-filename
  make[2]: Leaving directory `/d/gnu/git/emacs/trunk/lisp'
  Symbol's function definition is void: w32-convert-standard-filename
  make[2]: Leaving directory `/d/gnu/git/emacs/trunk/lisp'
  Directories for loaddefs: . ./calc ./calendar ./cedet ./cedet/ede ./cedet/semantic ./cedet/semantic/analyze ./cedet/semantic/bovine ./cedet/semantic/decorate ./cedet/semantic/symref ./cedet/semantic/wisent ./cedet/srecode ./emacs-lisp ./emulation ./erc ./eshell ./gnus ./image ./international ./language ./leim ./leim/ja-dic ./leim/quail ./mail ./mh-e ./net ./nxml ./org ./play ./progmodes ./textmodes ./url ./vc
    GEN      loaddefs.el
  Symbol's function definition is void: w32-convert-standard-filename

bug#25360: File mode specification errors during building

[martin rudalics]

 > I see multiple error messages about w32-convert-standard-filename
 > during a normal build:

I've seen them as well but eventually my build failed.  During the
subsequent bootstrap I don't recall seeing them.  BTW, in the failing
build I also saw some completely misspelled sentences with words like
"Fffuuunctiiioonss" or the like.  Queer.

martin

bug#25360: File mode specification errors during building

[Eli Zaretskii]

> Date: Thu, 02 Mar 2017 18:57:38 +0100
> From: martin rudalics <rudalics@gmx.at>
> CC: npostavs@users.sourceforge.net, 25360@debbugs.gnu.org
> 
> BTW, in the failing build I also saw some completely misspelled
> sentences with words like "Fffuuunctiiioonss" or the like.  Queer.

That happens with parallel builds, when more than one instance of byte
compilation displays the same text.  Quite normal.

bug#25360: File mode specification errors during building

[Eli Zaretskii]

> Date: Thu, 02 Mar 2017 17:20:52 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: npostavs@users.sourceforge.net, 25360@debbugs.gnu.org
> 
>   make[2]: Entering directory `/d/gnu/git/emacs/trunk/lisp'
>     ELC      ../lisp/files.elc
>     GEN      mh-e/mh-loaddefs.el
>   make[2]: Entering directory `/d/gnu/git/emacs/trunk/lisp'
>     ELC      ../lisp/international/mule.elc
>   Symbol's function definition is void: w32-convert-standard-filename
>   Symbol's function definition is void: w32-convert-standard-filename
>   make[2]: Leaving directory `/d/gnu/git/emacs/trunk/lisp'
>   Symbol's function definition is void: w32-convert-standard-filename
>   make[2]: Leaving directory `/d/gnu/git/emacs/trunk/lisp'
>   Directories for loaddefs: . ./calc ./calendar ./cedet ./cedet/ede ./cedet/semantic ./cedet/semantic/analyze ./cedet/semantic/bovine ./cedet/semantic/decorate ./cedet/semantic/symref ./cedet/semantic/wisent ./cedet/srecode ./emacs-lisp ./emulation ./erc ./eshell ./gnus ./image ./international ./language ./leim ./leim/ja-dic ./leim/quail ./mail ./mh-e ./net ./nxml ./org ./play ./progmodes ./textmodes ./url ./vc
>     GEN      loaddefs.el
>   Symbol's function definition is void: w32-convert-standard-filename

I still see these errors while building master.  Here's the latest
incarnation:

  Finding pointers to doc strings...
  Finding pointers to doc strings...done
  Dumping under the name emacs
  Dump memory usage: Heap: 9498624  Large blocks(0/1): 0/786448
  Dumping from d:/gnu/git/emacs/trunk/src/temacs.exe
	    to d:/gnu/git/emacs/trunk/src/emacs.exe
  49515 pure bytes used
  mv -f emacs.exe bootstrap-emacs.exe
  make -C ../lisp compile-first EMACS="../src/bootstrap-emacs.exe"
  make[2]: Entering directory `/d/gnu/git/emacs/trunk/lisp'
    GEN      mh-e/mh-loaddefs.el
  Symbol's function definition is void: w32-convert-standard-filename
  Directories for loaddefs: . ./calc ./calendar ./cedet ./cedet/ede ./cedet/semantic ./cedet/semantic/analyze ./cedet/semantic/bovine ./cedet/semantic/decorate ./cedet/semantic/symref ./cedet/semantic/wisent ./cedet/srecode ./emacs-lisp ./emulation ./erc ./eshell ./gnus ./image ./international ./language ./leim ./leim/ja-dic ./leim/quail ./mail ./mh-e ./net ./nxml ./org ./play ./progmodes ./textmodes ./url ./vc
    GEN      loaddefs.el
  Symbol's function definition is void: w32-convert-standard-filename
  make[2]: Leaving directory `/d/gnu/git/emacs/trunk/lisp'

The build succeeds nonetheless.  Moreover, if I repeat the offending
command ("make autoloads") manually, there are no error messages.
w32-convert-standard-filename is defined in w32-fns.el, which is
preloaded by loadup.el, so I don't understand this error message, and
am unable to debug it because I cannot reproduce if if I run the same
commands by hand.  It could be related to the fact that we use an
emacs executable that overflowed the pure storage (see the "49515 pure
bytes used" message).

Can we please fix this annoyance?  (And no, I don't want to bootstrap,
because this deletes all the old binaries I keep for testing.  And I
already tried "make generate-ldefs-boot", it didn't help.)

Thanks.

bug#25360: File mode specification errors during building

[Phillip Lord]

>>
>
> I still see these errors while building master.  Here's the latest
> incarnation:
>
>
> Finding pointers to doc strings...
> Finding pointers to doc strings...done
> Dumping under the name emacs
> Dump memory usage: Heap: 9498624  Large blocks(0/1): 0/786448
> Dumping from d:/gnu/git/emacs/trunk/src/temacs.exe
> to d:/gnu/git/emacs/trunk/src/emacs.exe 49515 pure bytes used
> mv -f emacs.exe bootstrap-emacs.exe make -C ../lisp compile-first
> EMACS="../src/bootstrap-emacs.exe"
> make[2]: Entering directory `/d/gnu/git/emacs/trunk/lisp'
> GEN      mh-e/mh-loaddefs.el
> Symbol's function definition is void: w32-convert-standard-filename
> Directories for loaddefs: . ./calc ./calendar ./cedet ./cedet/ede
> ./cedet/semantic ./cedet/semantic/analyze ./cedet/semantic/bovine
> ./cedet/semantic/decorate ./cedet/semantic/symref ./cedet/semantic/wisent
> ./cedet/srecode ./emacs-lisp ./emulation ./erc ./eshell ./gnus ./image
> ./international ./language ./leim ./leim/ja-dic ./leim/quail ./mail
> ./mh-e ./net ./nxml ./org ./play ./progmodes ./textmodes ./url ./vc
> GEN      loaddefs.el
> Symbol's function definition is void: w32-convert-standard-filename
> make[2]: Leaving directory `/d/gnu/git/emacs/trunk/lisp'
>
>
> The build succeeds nonetheless.  Moreover, if I repeat the offending
> command ("make autoloads") manually, there are no error messages.
> w32-convert-standard-filename is defined in w32-fns.el, which is preloaded
> by loadup.el, so I don't understand this error message, and am unable to
> debug it because I cannot reproduce if if I run the same commands by hand.
> It could be related to the fact that we use an
> emacs executable that overflowed the pure storage (see the "49515 pure
> bytes used" message).
>
> Can we please fix this annoyance?  (And no, I don't want to bootstrap,
> because this deletes all the old binaries I keep for testing.  And I
> already tried "make generate-ldefs-boot", it didn't help.)


Yes, I will revert to the old system. Nice idea, but it isn't working.
I'll try and find time as quickly as possible.

Phil

bug#25360: File mode specification errors during building

[Eli Zaretskii]

> Date: Sat, 4 Mar 2017 10:32:21 -0000
> From: "Phillip Lord" <phillip.lord@russet.org.uk>
> Cc: phillip.lord@russet.org.uk,
>  npostavs@users.sourceforge.net,
>  25360@debbugs.gnu.org
> 
> > Symbol's function definition is void: w32-convert-standard-filename
> > make[2]: Leaving directory `/d/gnu/git/emacs/trunk/lisp'
> >
> >
> > The build succeeds nonetheless.  Moreover, if I repeat the offending
> > command ("make autoloads") manually, there are no error messages.
> > w32-convert-standard-filename is defined in w32-fns.el, which is preloaded
> > by loadup.el, so I don't understand this error message, and am unable to
> > debug it because I cannot reproduce if if I run the same commands by hand.
> > It could be related to the fact that we use an
> > emacs executable that overflowed the pure storage (see the "49515 pure
> > bytes used" message).
> >
> > Can we please fix this annoyance?  (And no, I don't want to bootstrap,
> > because this deletes all the old binaries I keep for testing.  And I
> > already tried "make generate-ldefs-boot", it didn't help.)
> 
> 
> Yes, I will revert to the old system. Nice idea, but it isn't working.
> I'll try and find time as quickly as possible.

Thanks.  I'm making a few small changes in the current system, so
please wait for a while until I'm done.  If that doesn't solve the
above issue, at the very least it will fix a few minor gotcha's, in
case someone else will want to pick up this idea in the future.

bug#25360: File mode specification errors during building

[Eli Zaretskii]

> Date: Sat, 04 Mar 2017 13:11:16 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: npostavs@users.sourceforge.net, 25360@debbugs.gnu.org
> 
> > Yes, I will revert to the old system. Nice idea, but it isn't working.
> > I'll try and find time as quickly as possible.
> 
> Thanks.  I'm making a few small changes in the current system, so
> please wait for a while until I'm done.  If that doesn't solve the
> above issue, at the very least it will fix a few minor gotcha's, in
> case someone else will want to pick up this idea in the future.

I've committed those changes.  They didn't solve the problem.  I'm
quite sure the problem happens because we use an emacs executable
which overflowed the pure space when it was dumped, FWIW.

Thanks.

bug#25360: File mode specification errors during building

[Phillip Lord]

I've pushed the reversion to git.

http://git.savannah.gnu.org/cgit/emacs.git/log/?h=fix/great-revert-bill

Be happy if someone could try it and see if removes the problem.

Phil

bug#25360: File mode specification errors during building

[Noam Postavsky]

On Mon, Mar 6, 2017 at 10:33 AM, Phillip Lord
<phillip.lord@russet.org.uk> wrote:
>
>
> I've pushed the reversion to git.
>
> http://git.savannah.gnu.org/cgit/emacs.git/log/?h=fix/great-revert-bill
>
> Be happy if someone could try it and see if removes the problem.
>

This still fails on Windows with

  GEN      ../../lisp/net/tramp-loaddefs.el
Symbol's function definition is void: w32-convert-standard-filename
make[2]: *** [Makefile:404: ../../lisp/net/tramp-loaddefs.el] Error 127

After reverting 1b9463051823 "Remove conditional includes from
bootstrap" and deleting bootstrap-emacs.exe, I was able to build
successfully.

bug#25360: File mode specification errors during building

[Eli Zaretskii]

> From: Noam Postavsky <npostavs@users.sourceforge.net>
> Date: Mon, 6 Mar 2017 14:56:35 -0500
> Cc: Eli Zaretskii <eliz@gnu.org>, 25360@debbugs.gnu.org
> 
> This still fails on Windows with
> 
>   GEN      ../../lisp/net/tramp-loaddefs.el
> Symbol's function definition is void: w32-convert-standard-filename
> make[2]: *** [Makefile:404: ../../lisp/net/tramp-loaddefs.el] Error 127
> 
> After reverting 1b9463051823 "Remove conditional includes from
> bootstrap" and deleting bootstrap-emacs.exe, I was able to build
> successfully.

Hmm... I don't get it: why did that commit make Emacs bypass loading
important packages when invoked with "loadup bootstrap"?  Phillip, can
you explain the rationale?

bug#25360: File mode specification errors during building

[Noam Postavsky]

On Mon, Mar 6, 2017 at 3:53 PM, Eli Zaretskii <eliz@gnu.org> wrote:
>>
>> After reverting 1b9463051823 "Remove conditional includes from
>> bootstrap" and deleting bootstrap-emacs.exe, I was able to build
>> successfully.
>
> Hmm... I don't get it: why did that commit make Emacs bypass loading
> important packages when invoked with "loadup bootstrap"?  Phillip, can
> you explain the rationale?

It's discussed upthread, it has to do with making one of the ldefs
files consistent across configurations.

https://debbugs.gnu.org/cgi/bugreport.cgi?bug=25360#72
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=25360#93

It should not be needed now that we're going back to the old system, AFAIK.

bug#25360: File mode specification errors during building

[Phillip Lord]

On Mon, March 6, 2017 8:53 pm, Eli Zaretskii wrote:
>> From: Noam Postavsky <npostavs@users.sourceforge.net>
>> Date: Mon, 6 Mar 2017 14:56:35 -0500
>> Cc: Eli Zaretskii <eliz@gnu.org>, 25360@debbugs.gnu.org
>>
>>
>> This still fails on Windows with
>>
>>
>> GEN      ../../lisp/net/tramp-loaddefs.el
>> Symbol's function definition is void: w32-convert-standard-filename
>> make[2]: *** [Makefile:404: ../../lisp/net/tramp-loaddefs.el] Error 127
>>
>>
>> After reverting 1b9463051823 "Remove conditional includes from
>> bootstrap" and deleting bootstrap-emacs.exe, I was able to build
>> successfully.
>
> Hmm... I don't get it: why did that commit make Emacs bypass loading
> important packages when invoked with "loadup bootstrap"?  Phillip, can you
> explain the rationale?

Long story. Don't bother, it's going anyway, I just missed that one.

Here's a question, though. Why does files.el use
w32-convert-standard-filename, a function which is not actually
autoloaded? Ditto dos-convert-standard-filename.

bug#25360: File mode specification errors during building

[Eli Zaretskii]

> Date: Mon, 6 Mar 2017 21:25:54 -0000
> From: "Phillip Lord" <phillip.lord@russet.org.uk>
> Cc: "Noam Postavsky" <npostavs@users.sourceforge.net>,
>  phillip.lord@russet.org.uk,
>  25360@debbugs.gnu.org
> 
> Here's a question, though. Why does files.el use
> w32-convert-standard-filename, a function which is not actually
> autoloaded? Ditto dos-convert-standard-filename.

They are in files that are preloaded on the platforms which need them,
so they are always available.

bug#25360: File mode specification errors during building

[Phillip Lord]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Mon, 6 Mar 2017 21:25:54 -0000
>> From: "Phillip Lord" <phillip.lord@russet.org.uk>
>> Cc: "Noam Postavsky" <npostavs@users.sourceforge.net>,
>>  phillip.lord@russet.org.uk,
>>  25360@debbugs.gnu.org
>> 
>> Here's a question, though. Why does files.el use
>> w32-convert-standard-filename, a function which is not actually
>> autoloaded? Ditto dos-convert-standard-filename.
>
> They are in files that are preloaded on the platforms which need them,
> so they are always available.


Eech, three different ways of specifying dependencies between lisp
files? Ah, well, something for a different time.

Phil

bug#25360: File mode specification errors during building

[Phillip Lord]

phillip.lord@russet.org.uk (Phillip Lord) writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> Date: Mon, 6 Mar 2017 21:25:54 -0000
>>> From: "Phillip Lord" <phillip.lord@russet.org.uk>
>>> Cc: "Noam Postavsky" <npostavs@users.sourceforge.net>,
>>>  phillip.lord@russet.org.uk,
>>>  25360@debbugs.gnu.org
>>> 
>>> Here's a question, though. Why does files.el use
>>> w32-convert-standard-filename, a function which is not actually
>>> autoloaded? Ditto dos-convert-standard-filename.
>>
>> They are in files that are preloaded on the platforms which need them,
>> so they are always available.
>
>
> Eech, three different ways of specifying dependencies between lisp
> files? Ah, well, something for a different time.


I have reverted these changes on master now. I had to redo the
reversions in a different order to get a clean rebase against master,
which means its not quite the same thing.

I can't test this build against windows until later in the evening; it
did build fine before I reordered things, so I thought it reasonable to
commit.

Phil

bug#25360: File mode specification errors during building

[Eli Zaretskii]

> From: phillip.lord@russet.org.uk (Phillip Lord)
> Cc: npostavs@users.sourceforge.net,  25360@debbugs.gnu.org
> Date: Tue, 07 Mar 2017 15:28:44 +0000
> 
> I have reverted these changes on master now. I had to redo the
> reversions in a different order to get a clean rebase against master,
> which means its not quite the same thing.

Thanks.

> I can't test this build against windows until later in the evening; it
> did build fine before I reordered things, so I thought it reasonable to
> commit.

It builds OK for me on Windows, although I didn't (and won't) try
bootstrapping any time soon.

Thank you for your work on this, Phillip.

bug#25360: File mode specification errors during building

[Noam Postavsky]

On Tue, Mar 7, 2017 at 11:01 AM, Eli Zaretskii <eliz@gnu.org> wrote:
>
>> I can't test this build against windows until later in the evening; it
>> did build fine before I reordered things, so I thought it reasonable to
>> commit.
>
> It builds OK for me on Windows, although I didn't (and won't) try
> bootstrapping any time soon.
>

A bootstrap succeeded here (Windows 10, msys2, 64bit, out-of-tree).

bug#25360: File mode specification errors during building

[Phillip Lord]

On Tue, March 7, 2017 6:25 pm, Noam Postavsky wrote:
> On Tue, Mar 7, 2017 at 11:01 AM, Eli Zaretskii <eliz@gnu.org> wrote:
>
>>
>>> I can't test this build against windows until later in the evening;
>>> it did build fine before I reordered things, so I thought it
>>> reasonable to commit.
>>
>> It builds OK for me on Windows, although I didn't (and won't) try
>> bootstrapping any time soon.
>>
>
> A bootstrap succeeded here (Windows 10, msys2, 64bit, out-of-tree).

Also for me.

bug#25360: File mode specification errors during building

[Phillip Lord]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: phillip.lord@russet.org.uk (Phillip Lord)
>> Cc: npostavs@users.sourceforge.net,  25360@debbugs.gnu.org
>> Date: Tue, 07 Mar 2017 15:28:44 +0000
>> 
>> I have reverted these changes on master now. I had to redo the
>> reversions in a different order to get a clean rebase against master,
>> which means its not quite the same thing.
>
> Thanks.
>
>> I can't test this build against windows until later in the evening; it
>> did build fine before I reordered things, so I thought it reasonable to
>> commit.
>
> It builds OK for me on Windows, although I didn't (and won't) try
> bootstrapping any time soon.
>
> Thank you for your work on this, Phillip.


No worries. Was hoping that this would have made Emacs better, but it
doesn't, so it has to come out. I may revisit the issue after the
current discussions about dumping have been completed.

Phil

bug#25360: File mode specification errors during building

[Eli Zaretskii]

> From: phillip.lord@russet.org.uk (Phillip Lord)
> Cc: npostavs@users.sourceforge.net,  25360@debbugs.gnu.org
> Date: Tue, 07 Mar 2017 12:26:08 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Date: Mon, 6 Mar 2017 21:25:54 -0000
> >> From: "Phillip Lord" <phillip.lord@russet.org.uk>
> >> Cc: "Noam Postavsky" <npostavs@users.sourceforge.net>,
> >>  phillip.lord@russet.org.uk,
> >>  25360@debbugs.gnu.org
> >> 
> >> Here's a question, though. Why does files.el use
> >> w32-convert-standard-filename, a function which is not actually
> >> autoloaded? Ditto dos-convert-standard-filename.
> >
> > They are in files that are preloaded on the platforms which need them,
> > so they are always available.
> 
> 
> Eech, three different ways of specifying dependencies between lisp
> files? Ah, well, something for a different time.

It's not a dependency.  Emacs is full of code that doesn't bother to
auto-load functions which are known to be preloaded.  We preload Lisp
packages for several decades, and that's got to leave its impression
on the code which uses those packages.

bootstrap-emacs is not supposed to be different from emacs in that
regard, with a single exception: the Lisp packages it preloads might
be uncompiled, i.e. will work slower.  But it should have all the same
packages that emacs has.

bug#25360: File mode specification errors during building

[Andy Moreton]

On Sat 04 Mar 2017, Phillip Lord wrote:

>>>
>>
>> I still see these errors while building master.  Here's the latest
>> incarnation:
>>
>>
>> Finding pointers to doc strings...
>> Finding pointers to doc strings...done
>> Dumping under the name emacs
>> Dump memory usage: Heap: 9498624  Large blocks(0/1): 0/786448
>> Dumping from d:/gnu/git/emacs/trunk/src/temacs.exe
>> to d:/gnu/git/emacs/trunk/src/emacs.exe 49515 pure bytes used
>> mv -f emacs.exe bootstrap-emacs.exe make -C ../lisp compile-first
>> EMACS="../src/bootstrap-emacs.exe"
>> make[2]: Entering directory `/d/gnu/git/emacs/trunk/lisp'
>> GEN      mh-e/mh-loaddefs.el
>> Symbol's function definition is void: w32-convert-standard-filename
>> Directories for loaddefs: . ./calc ./calendar ./cedet ./cedet/ede
>> ./cedet/semantic ./cedet/semantic/analyze ./cedet/semantic/bovine
>> ./cedet/semantic/decorate ./cedet/semantic/symref ./cedet/semantic/wisent
>> ./cedet/srecode ./emacs-lisp ./emulation ./erc ./eshell ./gnus ./image
>> ./international ./language ./leim ./leim/ja-dic ./leim/quail ./mail
>> ./mh-e ./net ./nxml ./org ./play ./progmodes ./textmodes ./url ./vc
>> GEN      loaddefs.el
>> Symbol's function definition is void: w32-convert-standard-filename
>> make[2]: Leaving directory `/d/gnu/git/emacs/trunk/lisp'
>>
>>
>> The build succeeds nonetheless.  Moreover, if I repeat the offending
>> command ("make autoloads") manually, there are no error messages.
>> w32-convert-standard-filename is defined in w32-fns.el, which is preloaded
>> by loadup.el, so I don't understand this error message, and am unable to
>> debug it because I cannot reproduce if if I run the same commands by hand.
>> It could be related to the fact that we use an
>> emacs executable that overflowed the pure storage (see the "49515 pure
>> bytes used" message).
>>
>> Can we please fix this annoyance?  (And no, I don't want to bootstrap,
>> because this deletes all the old binaries I keep for testing.  And I
>> already tried "make generate-ldefs-boot", it didn't help.)
>
>
> Yes, I will revert to the old system. Nice idea, but it isn't working.
> I'll try and find time as quickly as possible.

Please do. I see the same messages about w32-convert-standard-filename,
but in my case the build always fails.

    AndyM

bug#25360: File mode specification errors during building

[Phillip Lord]

On Sun, March 5, 2017 6:26 pm, Andy Moreton wrote:
> On Sat 04 Mar 2017, Phillip Lord wrote:
>
>
>>>>
>>>
>>> I still see these errors while building master.  Here's the latest
>>> incarnation:
>>>
>>>
>>>
>>> Finding pointers to doc strings...
>>> Finding pointers to doc strings...done
>>> Dumping under the name emacs
>>> Dump memory usage: Heap: 9498624  Large blocks(0/1): 0/786448
>>> Dumping from d:/gnu/git/emacs/trunk/src/temacs.exe
>>> to d:/gnu/git/emacs/trunk/src/emacs.exe 49515 pure bytes used mv -f
>>> emacs.exe bootstrap-emacs.exe make -C ../lisp compile-first
>>> EMACS="../src/bootstrap-emacs.exe"
>>> make[2]: Entering directory `/d/gnu/git/emacs/trunk/lisp'
>>> GEN      mh-e/mh-loaddefs.el
>>> Symbol's function definition is void: w32-convert-standard-filename
>>> Directories for loaddefs: . ./calc ./calendar ./cedet ./cedet/ede
>>> ./cedet/semantic ./cedet/semantic/analyze ./cedet/semantic/bovine
>>> ./cedet/semantic/decorate ./cedet/semantic/symref
>>> ./cedet/semantic/wisent
>>> ./cedet/srecode ./emacs-lisp ./emulation ./erc ./eshell ./gnus ./image
>>>  ./international ./language ./leim ./leim/ja-dic ./leim/quail ./mail
>>> ./mh-e ./net ./nxml ./org ./play ./progmodes ./textmodes ./url ./vc
>>> GEN      loaddefs.el
>>> Symbol's function definition is void: w32-convert-standard-filename
>>> make[2]: Leaving directory `/d/gnu/git/emacs/trunk/lisp'
>>>
>>>
>>>
>>> The build succeeds nonetheless.  Moreover, if I repeat the offending
>>> command ("make autoloads") manually, there are no error messages.
>>> w32-convert-standard-filename is defined in w32-fns.el, which is
>>> preloaded by loadup.el, so I don't understand this error message, and
>>> am unable to debug it because I cannot reproduce if if I run the same
>>> commands by hand. It could be related to the fact that we use an
>>> emacs executable that overflowed the pure storage (see the "49515 pure
>>>  bytes used" message).
>>>
>>> Can we please fix this annoyance?  (And no, I don't want to
>>> bootstrap, because this deletes all the old binaries I keep for
>>> testing.  And I already tried "make generate-ldefs-boot", it didn't
>>> help.)
>>
>>
>> Yes, I will revert to the old system. Nice idea, but it isn't working.
>> I'll try and find time as quickly as possible.
>>
>
> Please do. I see the same messages about w32-convert-standard-filename,
> but in my case the build always fails.
>

In the mean time, try stuffing this:

(autoload 'w32-convert-standard-filename "w32-fns.el" nil nil nil)

into ldefs-boot-auto.el. generate-ldefs-boot adds this for me anyway. If
it works for you, I can add this to master quicker than I can revert.

Phil

bug#25360: File mode specification errors during building

[Eli Zaretskii]

> Date: Sun, 5 Mar 2017 18:58:52 -0000
> From: "Phillip Lord" <phillip.lord@russet.org.uk>
> Cc: 25360@debbugs.gnu.org
> 
> In the mean time, try stuffing this:
> 
> (autoload 'w32-convert-standard-filename "w32-fns.el" nil nil nil)
> 
> into ldefs-boot-auto.el. generate-ldefs-boot adds this for me anyway. If
> it works for you, I can add this to master quicker than I can revert.

Why would that change anything?  ldefs-boot-manual.el already includes
that line, so ldefs-boot-auto.el doesn't have to, right?

bug#25360: File mode specification errors during building

[Andy Moreton]

On Sun 05 Mar 2017, Eli Zaretskii wrote:

>> Date: Sun, 5 Mar 2017 18:58:52 -0000
>> From: "Phillip Lord" <phillip.lord@russet.org.uk>
>> Cc: 25360@debbugs.gnu.org
>> 
>> In the mean time, try stuffing this:
>> 
>> (autoload 'w32-convert-standard-filename "w32-fns.el" nil nil nil)
>> 
>> into ldefs-boot-auto.el. generate-ldefs-boot adds this for me anyway. If
>> it works for you, I can add this to master quicker than I can revert.
>
> Why would that change anything?  ldefs-boot-manual.el already includes
> that line, so ldefs-boot-auto.el doesn't have to, right?

Adding this to does not remove the build failures, so Eli is correct.

What is needed is to revert the changes that caused this regression in
the build machinery.

    AndyM

bug#25360: File mode specification errors during building

[Phillip Lord]

Andy Moreton <andrewjmoreton@gmail.com> writes:
>>> (autoload 'w32-convert-standard-filename "w32-fns.el" nil nil nil)
>>> 
>>> into ldefs-boot-auto.el. generate-ldefs-boot adds this for me anyway. If
>>> it works for you, I can add this to master quicker than I can revert.
>>
>> Why would that change anything?  ldefs-boot-manual.el already includes
>> that line, so ldefs-boot-auto.el doesn't have to, right?
>
> Adding this to does not remove the build failures, so Eli is correct.
>
> What is needed is to revert the changes that caused this regression in
> the build machinery.


That is on the branch that I emailed earlier. Will be happy to hear if
it solves the issue.

Phil

bug#25360: File mode specification errors during building

[Andy Moreton]

On Mon 06 Mar 2017, Phillip Lord wrote:

> Andy Moreton <andrewjmoreton@gmail.com> writes:
>>>> (autoload 'w32-convert-standard-filename "w32-fns.el" nil nil nil)
>>>> 
>>>> into ldefs-boot-auto.el. generate-ldefs-boot adds this for me anyway. If
>>>> it works for you, I can add this to master quicker than I can revert.
>>>
>>> Why would that change anything?  ldefs-boot-manual.el already includes
>>> that line, so ldefs-boot-auto.el doesn't have to, right?
>>
>> Adding this to does not remove the build failures, so Eli is correct.
>>
>> What is needed is to revert the changes that caused this regression in
>> the build machinery.
>
>
> That is on the branch that I emailed earlier. Will be happy to hear if
> it solves the issue.

I've tried building from a clean tree at commit be85e271 for 64bit
Windows (msys2). Bootstraps ok, and making trivial changes to C and lisp
files seems to also produce working builds thereafter.

    AndyM

bug#25360: File mode specification errors during building

[Phillip Lord]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Sun, 5 Mar 2017 18:58:52 -0000
>> From: "Phillip Lord" <phillip.lord@russet.org.uk>
>> Cc: 25360@debbugs.gnu.org
>> 
>> In the mean time, try stuffing this:
>> 
>> (autoload 'w32-convert-standard-filename "w32-fns.el" nil nil nil)
>> 
>> into ldefs-boot-auto.el. generate-ldefs-boot adds this for me anyway. If
>> it works for you, I can add this to master quicker than I can revert.
>
> Why would that change anything?  ldefs-boot-manual.el already includes
> that line, so ldefs-boot-auto.el doesn't have to, right?

Ah, yes, missed that, sorry.

Can't we just drop windows builds? That would make life easier.

Phil

bug#25360: File mode specification errors during building

[Eli Zaretskii]

> From: phillip.lord@russet.org.uk (Phillip Lord)
> Cc: andrewjmoreton@gmail.com,  25360@debbugs.gnu.org
> Date: Mon, 06 Mar 2017 16:30:19 +0000
> 
> Can't we just drop windows builds? That would make life easier.

We indeed intend to do that, on April 1st.

bug#25360: File mode specification errors during building

[Glenn Morris]

Thanks.
I can't decide whether modifying loadup.el for bootstrap-emacs in all
cases is good, bad, or neither... :)
Does it have any impact on the speed of builds, either way?
(Eg no longer loading something that wasn't needed;
or now autoloading every time something that used to be dumped.)

BTW, did you consider using a non-Emacs method to generate the basic loaddefs?
Eg maybe find + awk would be good enough to extract the necessary
autoloads to get a bootstrap going?

bug#25360: File mode specification errors during building

[Phillip Lord]

Glenn Morris <rgm@gnu.org> writes:

> Thanks.
> I can't decide whether modifying loadup.el for bootstrap-emacs in all
> cases is good, bad, or neither... :)

I worry about it slightly mostly because I'd rather not modify
loadup.el, especially given that the motivation is an error message that
isn't really causing any harm.

> Does it have any impact on the speed of builds, either way?

I checked whether my original modification made any difference, and the
answer is a few seconds. bootstrap-emacs is replaced by emacs after the
bootstrap build anyway (not sure why since actually, since I don't think
bootstrap-emacs is used again).

> (Eg no longer loading something that wasn't needed; or now autoloading
> every time something that used to be dumped.)

The former is true, but this *should* speed up the build (albeit only a
little). I don't think we are doing the latter, except for
'w32-convert-standard-filename and only on windows.


> BTW, did you consider using a non-Emacs method to generate the basic
> loaddefs?  Eg maybe find + awk would be good enough to extract the
> necessary autoloads to get a bootstrap going?

Yes, I did, but wasn't sure what extra dependencies the Emacs makefile
was allowed to have. I guess it already uses find and awk.

Also, I thought about building a bootstrap-emacs to generate loaddefs.el
(and nothing else), then another bootstrap-emacs to do what the current
one does. Or, split the loads and the bit which does the dump in
loadup.el and see if I could get temacs to do what bootstrap-emacs
currently does.

I was trying to be as conservative as possible, especially as the
dumping proceedure may change significantly soon. I guess my changes to
loadup.el do not qualify as that conservative now.

Phil

bug#25360: File mode specification errors during building

[Glenn Morris]

Phillip Lord wrote:

>> I can't decide whether modifying loadup.el for bootstrap-emacs in all
>> cases is good, bad, or neither... :)
>
> I worry about it slightly mostly because I'd rather not modify
> loadup.el, especially given that the motivation is an error message that
> isn't really causing any harm.

Yes, that was my concern too.

>> BTW, did you consider using a non-Emacs method to generate the basic
>> loaddefs?  Eg maybe find + awk would be good enough to extract the
>> necessary autoloads to get a bootstrap going?
>
> Yes, I did, but wasn't sure what extra dependencies the Emacs makefile
> was allowed to have. I guess it already uses find and awk.

Yes, find and (portable) awk is ok and already used eg for charsets
stuff.

bug#25360: File mode specification errors during building

[Eli Zaretskii]

> From: phillip.lord@russet.org.uk (Phillip Lord)
> Cc: rgm@gnu.org,  25360@debbugs.gnu.org
> Date: Fri, 13 Jan 2017 14:08:55 +0000
> 
> -  (search-backward "  GEN      loaddefs.el")
> +  (search-backward "-l loadup dump")
> +  (beginning-of-line)

Btw, relying on the likes of "  GEN      loaddefs.el" assumes the user
didn't say "make V=1", so I think it should be avoided anyway.

bug#25360: File mode specification errors during building

[Phillip Lord]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: phillip.lord@russet.org.uk (Phillip Lord)
>> Cc: rgm@gnu.org,  25360@debbugs.gnu.org
>> Date: Fri, 13 Jan 2017 14:08:55 +0000
>> 
>> -  (search-backward "  GEN      loaddefs.el")
>> +  (search-backward "-l loadup dump")
>> +  (beginning-of-line)
>
> Btw, relying on the likes of "  GEN      loaddefs.el" assumes the user
> didn't say "make V=1", so I think it should be avoided anyway.

Yeah, I know. I didn't worry too much about that in this case, since
this is an administrative task. This code is not run during a normal
bootstrap.

Phil

bug#25360: File mode specification errors during building

[Glenn Morris]

Phillip Lord wrote:

> The other would be just to stop generating CTLau.el and
> tstang-b5.el. Neither of the files from which they are generated have
> been materially changed in since 2001, and CTLau.html apparently does
> not exist any more in its source location. Deleting them, and adding
> CTLau.el as source would solve the problem and remove a step from the
> build.

For legal reasons, we need to distribute the "real" source.
(It also seems odd if the solution to a problem caused by removing a
generated file from the repository is... to add generated files to
the repository.)

bug#25360: File mode specification errors during building

[Phillip Lord]

Glenn Morris <rgm@gnu.org> writes:

> Phillip Lord wrote:
>
>> The other would be just to stop generating CTLau.el and
>> tstang-b5.el. Neither of the files from which they are generated have
>> been materially changed in since 2001, and CTLau.html apparently does
>> not exist any more in its source location. Deleting them, and adding
>> CTLau.el as source would solve the problem and remove a step from the
>> build.
>
> For legal reasons, we need to distribute the "real" source.

That's true, but not relevant in this case. The source is just the
preferred form for editing or updatable. Given that CTLau.html is no
longer being distributed from it's original source and is not,
therefore, updatable from that source, either form would work for
editing.

> (It also seems odd if the solution to a problem caused by removing a
> generated file from the repository is... to add generated files to
> the repository.)

No, that's not what I am suggesting. There is no point generating a
file from a source file if the source file never changes (it hasn't done
for 15 years) and if the generated file is always the same. It seems
worth doing independent of this bug.

Still, it's an independent issue, I shall see if I can figure the source
of the problem otherwise.

Phil

bug#25360: File mode specification errors during building

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > For legal reasons, we need to distribute the "real" source.

More than that, it is essential for _moral_ reasons.  A nonfree
program is an injustice.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

bug#25360: File mode specification errors during building

[Phillip Lord]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > For legal reasons, we need to distribute the "real" source.
>
> More than that, it is essential for _moral_ reasons.  A nonfree
> program is an injustice.

Also true, but not relevant in this case. I was suggesting changing the
"preferred form for making modifications" from one file (CTLau.html), to
another (CTLau.el). The latter is currently generated, but would become
source.

There is no legal nor moral reason to stop us from doing this. It's a
practical question of whether CTLau.html or CTLau.el is easier to
maintain.

Anyway, this is probably off-topic for this bug -- I shall open the
suggestion as a new bug.

Phil