Cleanup of byte-compiled files missing?

Cleanup of byte-compiled files missing?

[Lars Magne Ingebrigtsen]

I often `C-c' a "make", so the compilation is stopped in the middle of
compiling a file.  I don't recall this creating problems before, but now
I see I have lots of files like this:

  -rw-r--r-- 1 larsi larsi    937 Sep 17 03:45 rfc2045.elc
  lrwxrwxrwx 1 larsi larsi     40 Sep 14 22:22 .#rfc2045.elc12983Xpj -> larsi@quimbies.gnus.org.12983:1280527510
  -rw-r--r-- 1 larsi larsi    937 Sep 14 22:22 rfc2045.elc12983Xpj

Could this somehow be related to the recent change where
byte-compilation writes to a temporary file instead of directly to the
.elc file?

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: Cleanup of byte-compiled files missing?

[Glenn Morris]

Lars Magne Ingebrigtsen wrote:

> Could this somehow be related to the recent change where
> byte-compilation writes to a temporary file instead of directly to the
> .elc file?

Sure.

Re: Cleanup of byte-compiled files missing?

[Glenn Morris]

>   -rw-r--r-- 1 larsi larsi    937 Sep 14 22:22 rfc2045.elc12983Xpj

kill-emacs-hook is apparently now run on SIGTERM, but not in
batch-mode, so I don't know what to do about this.

Re: Cleanup of byte-compiled files missing?

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Date: Fri, 17 Sep 2010 13:41:05 -0400
> 
> 
> >   -rw-r--r-- 1 larsi larsi    937 Sep 14 22:22 rfc2045.elc12983Xpj
> 
> kill-emacs-hook is apparently now run on SIGTERM, but not in
> batch-mode, so I don't know what to do about this.

Here's a thought: make the temporary file and moving it to the .elc
file explicit in the Makefile, then you might be able to solve this
using Make features, at least on platforms that use GNU Make.  Make
automatically deletes targets whose processing was interrupted, and
you can use various special targets like .INTERMEDIATE and
.DELETE_ON_ERROR to tune and control this.

Re: Cleanup of byte-compiled files missing?

[Glenn Morris]

Eli Zaretskii wrote:

> Here's a thought: make the temporary file and moving it to the .elc
> file explicit in the Makefile, then you might be able to solve this
> using Make features, at least on platforms that use GNU Make.  Make
> automatically deletes targets whose processing was interrupted, and
> you can use various special targets like .INTERMEDIATE and
> .DELETE_ON_ERROR to tune and control this.

Good grief, that sounds like a lot of work. Plus it doesn't fix the
problem for people using Emacs to compile any other project.

Re: Cleanup of byte-compiled files missing?

[Glenn Morris]

Glenn Morris wrote:

> kill-emacs-hook is apparently now run on SIGTERM, but not in
> batch-mode,

I don't see why, BTW, since the doc of kill-emacs-hook explicitly says:

    Since `kill-emacs' may be invoked when the terminal is
    disconnected (or in other similar situations), functions placed on
    this hook should not expect to be able to interact with the user.

Re: Cleanup of byte-compiled files missing?

[Glenn Morris]

Here's a patch that seems to work, but I do not know if it is the
right thing to do. As well as running kill-emacs-hook in batch mode,
it is necessary to trap SIGINT.

Maybe it would be better to write the tempfiles to temporary-file-directory?

Still, if you interrupt make a lot, at least the current behaviour
must be better than the old, which would have left you wih
half-written .elc files.


=== modified file 'lisp/emacs-lisp/bytecomp.el'
*** lisp/emacs-lisp/bytecomp.el	2010-09-17 14:31:06 +0000
--- lisp/emacs-lisp/bytecomp.el	2010-09-20 22:44:37 +0000
***************
*** 1698,1709 ****
  	  (insert "\n")			; aaah, unix.
  	    (if (file-writable-p target-file)
  		;; We must disable any code conversion here.
! 		(let ((coding-system-for-write 'no-conversion)
! 		      ;; Write to a tempfile so that if another Emacs
! 		      ;; process is trying to load target-file (eg in a
! 		      ;; parallel bootstrap), it does not risk getting a
! 		      ;; half-finished file.  (Bug#4196)
! 		      (tempfile (make-temp-name target-file)))
  		  (if (memq system-type '(ms-dos 'windows-nt))
  		      (setq buffer-file-type t))
  		  (write-region (point-min) (point-max) tempfile nil 1)
--- 1698,1712 ----
  	  (insert "\n")			; aaah, unix.
  	    (if (file-writable-p target-file)
  		;; We must disable any code conversion here.
! 		(let* ((coding-system-for-write 'no-conversion)
! 		       ;; Write to a tempfile so that if another Emacs
! 		       ;; process is trying to load target-file (eg in a
! 		       ;; parallel bootstrap), it does not risk getting a
! 		       ;; half-finished file.  (Bug#4196)
! 		       (tempfile (make-temp-name target-file))
! 		       (kill-emacs-hook
! 			(cons (lambda () (ignore-errors (delete-file tempfile)))
! 			      kill-emacs-hook)))
  		  (if (memq system-type '(ms-dos 'windows-nt))
  		      (setq buffer-file-type t))
  		  (write-region (point-min) (point-max) tempfile nil 1)

=== modified file 'src/emacs.c'
*** src/emacs.c	2010-09-21 11:13:36 +0000
--- src/emacs.c	2010-09-21 20:31:42 +0000
***************
*** 374,380 ****
      {
        fatal_error_in_progress = 1;
  
!       if (sig == SIGTERM || sig == SIGHUP)
          Fkill_emacs (make_number (sig));
  
        shut_down_emacs (sig, 0, Qnil);
--- 374,380 ----
      {
        fatal_error_in_progress = 1;
  
!       if (sig == SIGTERM || sig == SIGHUP || sig == SIGINT)
          Fkill_emacs (make_number (sig));
  
        shut_down_emacs (sig, 0, Qnil);
***************
*** 1236,1241 ****
--- 1236,1242 ----
  #ifdef SIGSYS
        signal (SIGSYS, fatal_error_signal);
  #endif
+       if ( noninteractive ) signal (SIGINT, fatal_error_signal);
        signal (SIGTERM, fatal_error_signal);
  #ifdef SIGXCPU
        signal (SIGXCPU, fatal_error_signal);
***************
*** 1992,1998 ****
    if (feof (stdin))
      arg = Qt;
  
!   if (!NILP (Vrun_hooks) && !noninteractive)
      call1 (Vrun_hooks, intern ("kill-emacs-hook"));
  
    UNGCPRO;
--- 1993,1999 ----
    if (feof (stdin))
      arg = Qt;
  
!   if (!NILP (Vrun_hooks))
      call1 (Vrun_hooks, intern ("kill-emacs-hook"));
  
    UNGCPRO;
***************
*** 2413,2419 ****
  expect to be able to interact with the user.  To ask for confirmation,
  see `kill-emacs-query-functions' instead.
  
! The hook is not run in batch mode, i.e., if `noninteractive' is non-nil.  */);
    Vkill_emacs_hook = Qnil;
  
    DEFVAR_INT ("emacs-priority", &emacs_priority,
--- 2414,2421 ----
  expect to be able to interact with the user.  To ask for confirmation,
  see `kill-emacs-query-functions' instead.
  
! Before Emacs 24.1, the hook was not run in batch mode, i.e., if
! `noninteractive' was non-nil.  */);
    Vkill_emacs_hook = Qnil;
  
    DEFVAR_INT ("emacs-priority", &emacs_priority,

Re: Cleanup of byte-compiled files missing?

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Date: Tue, 21 Sep 2010 21:33:47 -0400
> 
> !       if (sig == SIGTERM || sig == SIGHUP || sig == SIGINT)
>           Fkill_emacs (make_number (sig));
>   
>         shut_down_emacs (sig, 0, Qnil);
> ***************
> *** 1236,1241 ****
> --- 1236,1242 ----
>   #ifdef SIGSYS
>         signal (SIGSYS, fatal_error_signal);
>   #endif
> +       if ( noninteractive ) signal (SIGINT, fatal_error_signal);

This will need to be "fixed" on MS-Windows, where a console program
interrupted with SIGINT runs the signal handler in a separate thread,
courtesy of MS runtime, which Emacs code doesn't expect.  If someone
wants to work on that, look in job.c from GNU Make, where I fixed a
similar problem.  Or maybe careful study of the issue will conclude
that running the handler in a separate thread is fine in Emacs, and
nothing special should be done.

Re: Cleanup of byte-compiled files missing?

[Davis Herring]

> Maybe it would be better to write the tempfiles to
> temporary-file-directory?

That wouldn't be enough: that directory might be on a different filesystem
(or even a different host) than the real elc, so you couldn't move it into
place atomically, and the original problem would recur if you copied it.

Davis

-- 
This product is sold by volume, not by mass.  If it appears too dense or
too sparse, it is because mass-energy conversion has occurred during
shipping.

Re: Cleanup of byte-compiled files missing?

[Glenn Morris]

Eli Zaretskii wrote:

> This will need to be "fixed" on MS-Windows,

I installed the relevant piece inside #ifndef WINDOWSNT.

Re: Cleanup of byte-compiled files missing?

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: Glenn Morris <rgm@gnu.org>
> Date: Fri, 01 Oct 2010 22:32:26 -0400
> 
> Eli Zaretskii wrote:
> 
> > This will need to be "fixed" on MS-Windows,
> 
> I installed the relevant piece inside #ifndef WINDOWSNT.

That's hardly what I meant.  "#ifndef WINDOWSNT" removed.

Re: Cleanup of byte-compiled files missing?

[Eli Zaretskii]

fatal_error_signal now says:

      if (sig == SIGTERM || sig == SIGHUP || sig == SIGINT)
        Fkill_emacs (make_number (sig));

Shouldn't the SIGINT part require noninteractive to be non-zero, just
to be on the safe side?

Re: Cleanup of byte-compiled files missing?

[Glenn Morris]

Eli Zaretskii wrote:

> fatal_error_signal now says:
>
>       if (sig == SIGTERM || sig == SIGHUP || sig == SIGINT)
>         Fkill_emacs (make_number (sig));
>
> Shouldn't the SIGINT part require noninteractive to be non-zero, just
> to be on the safe side?

I don't think it matters. It ought to be impossible to get there via
SIGINT except in batch mode, and if somehow Emacs does get there, it's
going to exit anyway, so it's just a question of if it goes via
kill_emacs (and so calls kill-emacs-hook) or straight to
shut_down_emacs ; and it seems to me it might as well run the hook as not.

Hmm, maybe the only real difference is the treatment of
Vauto_save_list_file_name.

Re: Cleanup of byte-compiled files missing?

[Glenn Morris]

Eli Zaretskii wrote:

>> > This will need to be "fixed" on MS-Windows,
>> 
>> I installed the relevant piece inside #ifndef WINDOWSNT.
>
> That's hardly what I meant.  "#ifndef WINDOWSNT" removed.

Sorry; I'm not interested in MS Windows so that was the easiest thing
for me.

Re: Cleanup of byte-compiled files missing?

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: Glenn Morris <rgm@gnu.org>
> Date: Sat, 02 Oct 2010 14:59:42 -0400
> 
> Eli Zaretskii wrote:
> 
> > fatal_error_signal now says:
> >
> >       if (sig == SIGTERM || sig == SIGHUP || sig == SIGINT)
> >         Fkill_emacs (make_number (sig));
> >
> > Shouldn't the SIGINT part require noninteractive to be non-zero, just
> > to be on the safe side?
> 
> I don't think it matters. It ought to be impossible to get there via
> SIGINT except in batch mode, and if somehow Emacs does get there, it's
> going to exit anyway, so it's just a question of if it goes via
> kill_emacs (and so calls kill-emacs-hook) or straight to
> shut_down_emacs ; and it seems to me it might as well run the hook as not.
> 
> Hmm, maybe the only real difference is the treatment of
> Vauto_save_list_file_name.

Why bother reasoning about that?  It's just one more flag to test, and
in a place where speed hardly matters at all.