bug#4616: 23.1.50; empty script files

bug#4616: 23.1.50; empty script files

[Markus Rost]

Please write in English if possible, because the Emacs maintainers
usually do not have translators to read other languages for them.

Your bug report will be posted to the emacs-pretest-bug@gnu.org mailing list.

Please describe exactly what actions triggered the bug
and the precise symptoms of the bug:


Execute an Emacs script file with no actual lisp code in it, like this:

===File ~/x.el==============================================
#!/home/rost/build/cvs/emacs/src/emacs --script

============================================================

One gets an error:

End of file during parsing: /home/rost/x.el

Same problem if the file has some additional ";"-commented lines in it.
No problem if there is a single not commented lisp expression in it,
like "()".

This patch fixes the problem for me:

*** lread.c	02 Oct 2009 14:05:54 +0200	1.414
--- lread.c	02 Oct 2009 14:11:36 +0200	
***************
*** 2613,2619 ****
  	     Skip the first line.  */
  	  while (c != '\n' && c >= 0)
  	    c = READCHAR;
! 	  goto retry;
  	}
        if (c == '$')
  	return Vload_file_name;
--- 2613,2619 ----
  	     Skip the first line.  */
  	  while (c != '\n' && c >= 0)
  	    c = READCHAR;
! 	  return Qnil;
  	}
        if (c == '$')
  	return Vload_file_name;


If Emacs crashed, and you have the Emacs process in the gdb debugger,
please include the output from the following gdb commands:
    `bt full' and `xbacktrace'.
If you would like to further debug the crash, please read the file
/home/rost/build/cvs/emacs/etc/DEBUG for instructions.


In GNU Emacs 23.1.50.1 (i686-pc-linux-gnu, GTK+ Version 2.16.1)
 of 2009-10-02 on laptop
Windowing system distributor `The X.Org Foundation', version 11.0.10600000
configured using `configure  '--prefix=/home/rost/local/cvs''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: C
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: nil
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: Fundamental

Minor modes in effect:
  tooltip-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  blink-cursor-mode: t
  global-auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
<help-echo> <help-echo> <help-echo> <down-mouse-1> 
<mouse-1> M-x r e p SPC o r SPC e m SPC SPC SPC <r
eturn>

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Making completion list...

Load-path shadows:
None found.

bug#4616: 23.1.50; empty script files

[Markus Rost]

What about the following tentative patch?

It moves the handling of '#!' from read1 up to readevalloop, which
then would skip the first line if it starts with '#!'.

To achieve this, the patch allows the readchar/unreadchar mechanism to
look 2 characters ahead, instead of just 1.

Omitting ' && first_sexp' in the patch would skip '#!' + rest of line
anywhere as before.

Be aware that I just made some guesses after glancing briefly over
lread.c (and I am not a C-programmer).


*** /home/rost/build/cvs/emacs/src/lread.c.~1.414~	2009-10-05 01:38:28.000000000 +0200
--- /home/rost/build/cvs/emacs/src/lread.c	2009-10-05 04:31:51.000000000 +0200
***************
*** 271,276 ****
--- 271,277 ----
     a file stream can't handle multibyte-char unreading.  The value -1
     means that there's no unread character. */
  static int unread_char;
+ static int unread_char1;
  
  static int
  readchar (readcharfun, multibyte)
***************
*** 410,415 ****
--- 411,422 ----
    return XINT (tem);
  
   read_multibyte:
+   if (unread_char1 >= 0)
+     {
+       c = unread_char1;
+       unread_char1 = -1;
+       return c;
+     }
    if (unread_char >= 0)
      {
        c = unread_char;
***************
*** 489,499 ****
      }
    else if (CONSP (readcharfun))
      {
!       unread_char = c;
      }
    else if (EQ (readcharfun, Qlambda))
      {
!       unread_char = c;
      }
    else if (EQ (readcharfun, Qget_file_char)
  	   || EQ (readcharfun, Qget_emacs_mule_file_char))
--- 496,512 ----
      }
    else if (CONSP (readcharfun))
      {
!       if (unread_char >= 0)
! 	unread_char1 = c;
!       else
! 	unread_char = c;
      }
    else if (EQ (readcharfun, Qlambda))
      {
!       if (unread_char >= 0)
! 	unread_char1 = c;
!       else
! 	unread_char = c;
      }
    else if (EQ (readcharfun, Qget_file_char)
  	   || EQ (readcharfun, Qget_emacs_mule_file_char))
***************
*** 505,511 ****
  	  UNBLOCK_INPUT;
  	}
        else
! 	unread_char = c;
      }
    else
      call1 (readcharfun, make_number (c));
--- 518,527 ----
  	  UNBLOCK_INPUT;
  	}
        else
! 	if (unread_char >= 0)
! 	  unread_char1 = c;
! 	else
! 	  unread_char = c;
      }
    else
      call1 (readcharfun, make_number (c));
***************
*** 1658,1663 ****
--- 1674,1680 ----
       Lisp_Object start, end;
  {
    register int c;
+   register int c1;
    register Lisp_Object val;
    int count = SPECPDL_INDEX ();
    struct gcpro gcpro1, gcpro2, gcpro3, gcpro4;
***************
*** 1747,1752 ****
--- 1764,1782 ----
  	  unbind_to (count1, Qnil);
  	  break;
  	}
+       if (c == '#' && first_sexp)
+ 	{
+ 	  c1 = READCHAR;
+ 	  if (c1 == '!')
+ 	    {
+ 	      /* #! appears at the beginning of an executable file.
+ 		 Skip the first line.  */
+ 	      while ((c1 = READCHAR) != '\n' && c1 != -1);
+ 	      goto read_next;
+ 	    }
+ 	  else
+ 	    UNREAD (c1);
+ 	}
  
        /* Ignore whitespace here, so we can detect eof.  */
        if (c == ' ' || c == '\t' || c == '\n' || c == '\f' || c == '\r'
***************
*** 2607,2620 ****
  	  load_each_byte = 0;
  	  goto retry;
  	}
-       if (c == '!')
- 	{
- 	  /* #! appears at the beginning of an executable file.
- 	     Skip the first line.  */
- 	  while (c != '\n' && c >= 0)
- 	    c = READCHAR;
- 	  goto retry;
- 	}
        if (c == '$')
  	return Vload_file_name;
        if (c == '\'')
--- 2637,2642 ----

bug#4616: 23.1.50; empty script files

[Markus Rost]

One more thing about unread_char in lread.c.  That variable should
start with value -1 rather than value 0.  The current initial value 0
means that the character 0 is added in front of loadup.el when
building emacs.  Now since loadup.el isn't empty, this doesn't cause
a read error, the additional character will be just skipped and the
value of unread_char is set to -1 after loading loadup.el.

So one doesn't notice the wrong initial value.  But maybe one changes
that for clarity.


*** lread.c	06 Oct 2009 14:22:15 +0200	1.414
--- lread.c	06 Oct 2009 17:01:06 +0200	
***************
*** 270,276 ****
     Qlambda, or a cons, we use this to keep an unread character because
     a file stream can't handle multibyte-char unreading.  The value -1
     means that there's no unread character. */
! static int unread_char;
  
  static int
  readchar (readcharfun, multibyte)
--- 270,276 ----
     Qlambda, or a cons, we use this to keep an unread character because
     a file stream can't handle multibyte-char unreading.  The value -1
     means that there's no unread character. */
! static int unread_char = -1;
  
  static int
  readchar (readcharfun, multibyte)

bug#4616: 23.1.50; empty script files

[Markus Rost]

What is the best way to implement --script?  Let me discuss some
possibilities:

1. The current implementation.  It consists of the lines

      if (c == '!')
	{
	  /* #! appears at the beginning of an executable file.
	     Skip the first line.  */
	  while (c != '\n' && c >= 0)
	    c = READCHAR;
	  goto retry;
	}

   in function read1 in lread.c.  This is appealing by its brevity.
   However it has the disadvantage that the basic read function has a
   *general* feature in order to get --script and the "#!"-syntax
   working.  Further it leads to the bug originally posted:  Execution
   of an empty script file like

   ===File ~/x.el==============================================
   #!/home/rost/build/cvs/emacs/src/emacs --script
   ============================================================

   yields an "End of file during parsing" error and returns with positive
   exit status.

2. One may think of handling --script by first removing a first
   #!-line before passing the file to load.  This seems to be very
   hard to implement because the load routine is complicated
   (switching back and forth between C and Elisp functions).  Besides
   this practical problem, there is the following disadvantage:  In
   some circumstances one may want to load a script file directly, as
   in

   emacs -batch ... -l ~/x.el ...

   Moreover, when editing the file one may want to test it with
   eval-buffer.

3. This leaves the possibility to handle #!-lines when running
   eval-region.  It should skip a #!-line at the beginning of the
   region.  It could skip any #!-line in the region, which is what
   read1 does right now, but I see no necessity for that.  This would
   be implemented by my two previous patches

   <URL:http://lists.gnu.org/archive/html/bug-gnu-emacs/2009-10/msg00082.html>
   <URL:http://lists.gnu.org/archive/html/bug-gnu-emacs/2009-10/msg00124.html>

   Of course making a change in readevalloop/read1 needs a careful
   check, maybe by several people.  However, the patch is a very
   "local" one around the variable unread_char and there are only a
   few places where unread_char appears, so checking the patch is not
   too much work.  Moreover, if something would be wrong with the
   change, very likely emacs would not even compile as many files are
   loaded during the build process.

bug#4616: 23.1.50; empty script files

[Lars Ingebrigtsen]

Markus Rost <rost@math.uni-bielefeld.de> writes:

>> The Emacs Lisp reader always requires at least one sexp in the input.
>> This is independent of the source of the input.
>
> Oh yes.  My patch would violate this.  But the Emacs Lisp "loader"
> doesn't require any input.
>
> I think, somebody without knowledge of Emacs internals will be
> surprised that
>
> ===File ~/x.el==============================================
> #!/home/rost/build/cvs/emacs/src/emacs --script
> ;; (some-valid-elisp-expression)
> ============================================================
>
> yields an "End of file during parsing" error and returns with positive
> exit status.

It's not quite clear to me what the issue here is.

--script requires at least one readable form in the script, and that
seems OK to me.  Is the problem the exit status?

> And I think, somebody executing pure elisp will be surprised that
>
> (read "#!\n(foo)")
>
> does not yield an "Invalid read syntax" error.
>
> It seems that the basic read function has a *general* feature in order
> to get --script and the "#!"-syntax working.

Heh.  That is quite amusing...

But can we change that at this point?  Presumably Emacs has had this
quirk for generations, so perhaps there's some code out there that
relies on the accidental comment-ness of #!...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#4616: 23.1.50; empty script files

[Markus Rost]

> --script requires at least one readable form in the script, and that
> seems OK to me.

A script with no commands should be valid in any language, in my
opinion.  As a matter of principle, but also for practical reasons:
You may want to comment by chance all code.  I think this way the
matter appeared to me.  You may also want to write a template with all
code commented initially.

> Is the problem the exit status?

I don't think that was my issue.

> But can we change that at this point?

Frightening prospect...

bug#4616: 23.1.50; empty script files

[Lars Ingebrigtsen]

Markus Rost <rost@math.uni-bielefeld.de> writes:

>> --script requires at least one readable form in the script, and that
>> seems OK to me.
>
> A script with no commands should be valid in any language, in my
> opinion.  As a matter of principle, but also for practical reasons:
> You may want to comment by chance all code.  I think this way the
> matter appeared to me.  You may also want to write a template with all
> code commented initially.

Sure, that's true.

I think the way to fix this is to tell the reader that we're reading
from a script file and then do the appropriate thing instead of altering
the reader unconditionally.  That is,

  if (argmatch (argv, argc, "-script", "--script", 3, &junk, &skip_args))
    {
      noninteractive = 1;	/* Set batch mode.  */

in emacs.c should set some global variable, and then readevalloop (I
think?) shouldn't signal an error if it turns out that there are no
forms to read.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#4616: 23.1.50; empty script files

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> I think the way to fix this is to tell the reader that we're reading
> from a script file and then do the appropriate thing instead of altering
> the reader unconditionally.  That is,
>
>   if (argmatch (argv, argc, "-script", "--script", 3, &junk, &skip_args))
>     {
>       noninteractive = 1;	/* Set batch mode.  */
>
> in emacs.c should set some global variable, and then readevalloop (I
> think?) shouldn't signal an error if it turns out that there are no
> forms to read.

I implemented this a different way in Emacs 29, so this problem has now
been fixed.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no