* bug#48940: 27.2; regression: "emacs --script /dev/stdin" parses the script incorrectly when /dev/stdin is a pipe @ 2021-06-09 20:31 Bryan C. Mills via Bug reports for GNU Emacs, the Swiss army knife of text editors 2021-06-13 10:26 ` Eli Zaretskii 0 siblings, 1 reply; 6+ messages in thread From: Bryan C. Mills via Bug reports for GNU Emacs, the Swiss army knife of text editors @ 2021-06-09 20:31 UTC (permalink / raw) To: 48940 `emacs --script /dev/stdin` worked reliably for me on Emacs 24 through 26. As of Emacs 27, it no longer works reliably — depending on the script contents, it either fails with spurious errors or silently exits without finishing the script. In the following bash interaction, all three emacs invocations *should* run the same script. The third invocation demonstrates the bug: when /dev/stdin is a pipe from `cat` (instead of the script.el file directly), the script is no longer interpreted. With other scripts, I get various errors, seemingly due to `;` or `"` characters being dropped or ignored from the input. ``` $ cat script.el | cat /dev/stdin (print "Hello emacs!") $ emacs --no-init-file --no-site-file --script script.el "Hello emacs!" $ emacs --no-init-file --no-site-file --script /dev/stdin <script.el "Hello emacs!" $ cat script.el | emacs --no-init-file --no-site-file --script /dev/stdin $ echo $? 0 $ ``` This reproduces with the GNU Emacs 27.1 currently packaged on Debian Testing, as well as the Google build of GNU Emacs 27.2 from which I produced the report below. (My apologies if this is a duplicate report. I tried to send it earlier using M-x report-emacs-bug, but the resulting email appears to have gotten stuck or rejected somewhere.) In GNU Emacs 27.2 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.24, cairo version 1.16.0) of 2021-03-29, modified by Debian built on kokoro-ubuntu System Description: Debian GNU/Linux rodete Recent messages: For information about GNU Emacs and the GNU system, type C-h C-a. <f6> is undefined [2 times] Making completion list... [2 times] Configured using: 'configure --build x86_64-linux-gnu --build x86_64-linux-gnu --prefix=/usr --sharedstatedir=/var/lib --libexecdir=/usr/lib --localstatedir=/var/lib --infodir=/usr/share/info --mandir=/usr/share/man --enable-libsystemd --with-pop=yes --enable-locallisppath=/etc/google-emacs:/usr/local/share/google-emacs/27.2+gg3+1.20210329.053400.rc147/site-lisp:/usr/local/share/google-emacs/site-lisp:/usr/share/google-emacs/27.2+gg3+1.20210329.053400.rc147/site-lisp:/usr/share/google-emacs/site-lisp --with-sound=alsa --without-gconf --with-mailutils --program-prefix=google- --disable-silent-rules GOOGLE_VERSION=27.2+gg3+1.20210329.053400.rc147 --with-cairo --with-x=yes --with-x-toolkit=gtk3 --with-toolkit-scroll-bars build_alias=x86_64-linux-gnu 'CFLAGS=-g -O2 -ffile-prefix-map=/build/google-emacs-avRD2q/google-emacs-27.2+gg3+1.20210329.053400.rc147=. -fstack-protector-strong -Wformat -Werror=format-security -Wall' LDFLAGS=-Wl,-z,relro 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' 'OBJCFLAGS=-g -O2 -ffile-prefix-map=/build/google-emacs-avRD2q/google-emacs-27.2+gg3+1.20210329.053400.rc147=. -fstack-protector-strong -Wformat -Werror=format-security'' Configured features: XPM JPEG TIFF GIF PNG RSVG CAIRO SOUND GPM DBUS GSETTINGS GLIB NOTIFY INOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES THREADS LIBSYSTEMD JSON PDUMPER LCMS2 GMP Important settings: value of $LC_ALL: en_US.UTF-8 value of $LANG: en_US.UTF-8 locale-coding-system: utf-8-unix Major mode: Text Minor modes in effect: tooltip-mode: t global-eldoc-mode: t electric-indent-mode: t mouse-wheel-mode: t tool-bar-mode: t menu-bar-mode: t file-name-shadow-mode: t global-font-lock-mode: t font-lock-mode: t auto-composition-mode: t auto-encryption-mode: t auto-compression-mode: t line-number-mode: t transient-mark-mode: t Load-path shadows: None found. Features: (shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs format-spec rfc822 mml mml-sec password-cache epa derived epg epg-config gnus-util rmail rmail-loaddefs text-property-search time-date subr-x seq mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils vc-git diff-mode easymenu easy-mmode cl-loaddefs cl-lib term/tmux term/xterm xterm byte-opt gv bytecomp byte-compile cconv tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type mwheel term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe tabulated-list replace newcomment text-mode elisp-mode lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch timer select scroll-bar mouse jit-lock font-lock syntax facemenu font-core term/tty-colors frame minibuffer cl-generic cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european ethiopic indian cyrillic chinese composite charscript charprop case-table epa-hook jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice loaddefs button faces cus-face macroexp files text-properties overlay sha1 md5 base64 format env code-pages mule custom widget hashtable-print-readable backquote threads dbusbind inotify lcms2 dynamic-setting system-font-setting font-render-setting cairo move-toolbar gtk x-toolkit x multi-tty make-network-process emacs) Memory information: ((conses 16 53224 7924) (symbols 48 6747 1) (strings 32 17548 1920) (string-bytes 1 565159) (vectors 16 8089) (vector-slots 8 89351 7266) (floats 8 29 624) (intervals 56 241 3) (buffers 1000 13)) ^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#48940: 27.2; regression: "emacs --script /dev/stdin" parses the script incorrectly when /dev/stdin is a pipe 2021-06-09 20:31 bug#48940: 27.2; regression: "emacs --script /dev/stdin" parses the script incorrectly when /dev/stdin is a pipe Bryan C. Mills via Bug reports for GNU Emacs, the Swiss army knife of text editors @ 2021-06-13 10:26 ` Eli Zaretskii 2021-06-14 21:37 ` Bryan C. Mills via Bug reports for GNU Emacs, the Swiss army knife of text editors 0 siblings, 1 reply; 6+ messages in thread From: Eli Zaretskii @ 2021-06-13 10:26 UTC (permalink / raw) To: Bryan C. Mills; +Cc: 48940 > Date: Wed, 9 Jun 2021 16:31:36 -0400 > From: "Bryan C. Mills" via "Bug reports for GNU Emacs, > the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org> > > `emacs --script /dev/stdin` worked reliably for me on Emacs 24 through 26. > > As of Emacs 27, it no longer works reliably — depending on the script contents, > it either fails with spurious errors or silently exits without > finishing the script. > > > In the following bash interaction, all three emacs invocations > *should* run the same script. > The third invocation demonstrates the bug: when /dev/stdin is a pipe from `cat` > (instead of the script.el file directly), the script is no longer interpreted. > With other scripts, I get various errors, seemingly due to `;` or `"` > characters being > dropped or ignored from the input. > > ``` > $ cat script.el | cat /dev/stdin > (print "Hello emacs!") > > $ emacs --no-init-file --no-site-file --script script.el > > "Hello emacs!" > > $ emacs --no-init-file --no-site-file --script /dev/stdin <script.el > > "Hello emacs!" > > $ cat script.el | emacs --no-init-file --no-site-file --script /dev/stdin > > $ echo $? > 0 > > $ > ``` > > This reproduces with the GNU Emacs 27.1 currently packaged on Debian Testing, > as well as the Google build of GNU Emacs 27.2 from which I produced > the report below. I see the problem, but it happens for me in Emacs 26 and Emacs 25 as well, so I'm not sure this is new, or why it works for you in older versions. Maybe Debian included some local patches in those older versions? (My old Emacs versions were built from the unmodified upstream sources.) AFAICS, the problem seems to be that load-with-code-conversion calls insert-file-contents, and the latter comes up with an empty buffer in the problematic case. It is notoriously hard to debug a program whose standard input was redirected from a pipe of another program, so I couldn't see why the above happens. If someone could step in a debugger through insert-file-contents in this case and describe what's going on there, or tell how to do that when Emacs is invoked like this, maybe we could make some progress. Thanks. ^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#48940: 27.2; regression: "emacs --script /dev/stdin" parses the script incorrectly when /dev/stdin is a pipe 2021-06-13 10:26 ` Eli Zaretskii @ 2021-06-14 21:37 ` Bryan C. Mills via Bug reports for GNU Emacs, the Swiss army knife of text editors 2021-11-11 5:09 ` Lars Ingebrigtsen 0 siblings, 1 reply; 6+ messages in thread From: Bryan C. Mills via Bug reports for GNU Emacs, the Swiss army knife of text editors @ 2021-06-14 21:37 UTC (permalink / raw) To: Eli Zaretskii; +Cc: 48940 I did a little digging and found that I can reproduce the issue under a debugger by constructing a named pipe with `mkfifo pipe.el` and feeding the script into it using `cat script.el >>pipe.el`. (Then I can execute `run --script pipe.el` under gdb to capture a backtrace at the problematic lseek call.) I identified one bug in the safe_to_load_version function. I think I've got a fix for it, but I'm not very familiar with the emacs codebase, especially when it comes to regression testing. (My patch against the emacs-27 branch is below.) I tested the fix locally with a small dummy script as input and it seems to work, but I can't be confident that there won't be deeper bugs. You're welcome to use it as a starting point for a more robust fix. From c97e4b97af86023177417e49c46b19812b2f4caa Mon Sep 17 00:00:00 2001 From: "Bryan C. Mills" <bcmills@google.com> Date: Mon, 14 Jun 2021 17:30:11 -0400 Subject: [PATCH] Fix invocation with '--script /dev/stdin' * src/lread.c (safe_to_load_version): Check lseek errors. (Bug#48940) --- src/lread.c | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-) diff --git a/src/lread.c b/src/lread.c index 47116ad5ae..96b6695b5c 100644 --- a/src/lread.c +++ b/src/lread.c @@ -990,12 +990,21 @@ #define UPDATE_BEG_END_STATE(ch) \ because of an incompatible change in the byte compiler. */ static int -safe_to_load_version (int fd) +safe_to_load_version (Lisp_Object file, int fd) { + struct stat st; char buf[512]; int nbytes, i; int version = 1; + /* If the file is not regular, then we cannot safely seek it. + Assume that it is not safe to load as a compiled file. */ + if (fstat(fd, &st) == 0) + { + if (!S_ISREG (st.st_mode)) + return 0; + } + /* Read the first few bytes from the file, and look for a line specifying the byte compiler version used. */ nbytes = emacs_read_quit (fd, buf, sizeof buf); @@ -1013,7 +1022,9 @@ safe_to_load_version (int fd) version = 0; } - lseek (fd, 0, SEEK_SET); + if (lseek (fd, 0, SEEK_SET) < 0) + report_file_error ("Seeking to start of file", file); + return version; } @@ -1316,7 +1327,7 @@ DEFUN ("load", Fload, Sload, 1, 5, 0, if (is_elc /* version = 1 means the file is empty, in which case we can treat it as not byte-compiled. */ - || (fd >= 0 && (version = safe_to_load_version (fd)) > 1)) + || (fd >= 0 && (version = safe_to_load_version (file, fd)) > 1)) /* Load .elc files directly, but not when they are remote and have no handler! */ { @@ -1326,7 +1337,7 @@ DEFUN ("load", Fload, Sload, 1, 5, 0, int result; if (version < 0 - && ! (version = safe_to_load_version (fd))) + && ! (version = safe_to_load_version (file, fd))) { safe_p = 0; if (!load_dangerous_libraries) -- 2.32.0.272.g935e593368-goog On Sun, Jun 13, 2021 at 6:26 AM Eli Zaretskii <eliz@gnu.org> wrote: > > > Date: Wed, 9 Jun 2021 16:31:36 -0400 > > From: "Bryan C. Mills" via "Bug reports for GNU Emacs, > > the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org> > > > > `emacs --script /dev/stdin` worked reliably for me on Emacs 24 through 26. > > > > As of Emacs 27, it no longer works reliably — depending on the script contents, > > it either fails with spurious errors or silently exits without > > finishing the script. > > > > > > In the following bash interaction, all three emacs invocations > > *should* run the same script. > > The third invocation demonstrates the bug: when /dev/stdin is a pipe from `cat` > > (instead of the script.el file directly), the script is no longer interpreted. > > With other scripts, I get various errors, seemingly due to `;` or `"` > > characters being > > dropped or ignored from the input. > > > > ``` > > $ cat script.el | cat /dev/stdin > > (print "Hello emacs!") > > > > $ emacs --no-init-file --no-site-file --script script.el > > > > "Hello emacs!" > > > > $ emacs --no-init-file --no-site-file --script /dev/stdin <script.el > > > > "Hello emacs!" > > > > $ cat script.el | emacs --no-init-file --no-site-file --script /dev/stdin > > > > $ echo $? > > 0 > > > > $ > > ``` > > > > This reproduces with the GNU Emacs 27.1 currently packaged on Debian Testing, > > as well as the Google build of GNU Emacs 27.2 from which I produced > > the report below. > > I see the problem, but it happens for me in Emacs 26 and Emacs 25 as > well, so I'm not sure this is new, or why it works for you in older > versions. Maybe Debian included some local patches in those older > versions? (My old Emacs versions were built from the unmodified > upstream sources.) > > AFAICS, the problem seems to be that load-with-code-conversion calls > insert-file-contents, and the latter comes up with an empty buffer in > the problematic case. > > It is notoriously hard to debug a program whose standard input was > redirected from a pipe of another program, so I couldn't see why the > above happens. If someone could step in a debugger through > insert-file-contents in this case and describe what's going on there, > or tell how to do that when Emacs is invoked like this, maybe we > could make some progress. > > Thanks. ^ permalink raw reply related [flat|nested] 6+ messages in thread
* bug#48940: 27.2; regression: "emacs --script /dev/stdin" parses the script incorrectly when /dev/stdin is a pipe 2021-06-14 21:37 ` Bryan C. Mills via Bug reports for GNU Emacs, the Swiss army knife of text editors @ 2021-11-11 5:09 ` Lars Ingebrigtsen 2021-11-11 17:23 ` Bryan C. Mills via Bug reports for GNU Emacs, the Swiss army knife of text editors 0 siblings, 1 reply; 6+ messages in thread From: Lars Ingebrigtsen @ 2021-11-11 5:09 UTC (permalink / raw) To: Bryan C. Mills; +Cc: 48940 "Bryan C. Mills" <bcmills@google.com> writes: > I tested the fix locally with a small dummy script as input and it > seems to work, but I can't be confident that there won't be deeper > bugs. You're welcome to use it as a starting point for a more robust > fix. I tried (on Debian/bullseye with Emacs 29), but it doesn't seem to make any difference here. With or without the patch, I get: larsi@xo:~/src/emacs/trunk$ emake; cat /tmp/script.el | ./src/emacs -Q --script /dev/stdin Debugger entered--Lisp error: (file-missing "Cannot open load file" "No such file or directory" "/proc/162231/fd/pipe:[1065564]") load("/proc/162231/fd/pipe:[1065564]" nil t t) command-line-1(("-scriptload" "/dev/stdin")) command-line() normal-top-level() Perhaps something further has changed in the meantime? (You didn't post an output of the error message you were getting...) -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no ^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#48940: 27.2; regression: "emacs --script /dev/stdin" parses the script incorrectly when /dev/stdin is a pipe 2021-11-11 5:09 ` Lars Ingebrigtsen @ 2021-11-11 17:23 ` Bryan C. Mills via Bug reports for GNU Emacs, the Swiss army knife of text editors 2021-11-12 3:26 ` Lars Ingebrigtsen 0 siblings, 1 reply; 6+ messages in thread From: Bryan C. Mills via Bug reports for GNU Emacs, the Swiss army knife of text editors @ 2021-11-11 17:23 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: Eli Zaretskii, 48940 On Thu, Nov 11, 2021 at 12:09 AM Lars Ingebrigtsen <larsi@gnus.org> wrote: > > "Bryan C. Mills" <bcmills@google.com> writes: > > > I tested the fix locally with a small dummy script as input and it > > seems to work, but I can't be confident that there won't be deeper > > bugs. You're welcome to use it as a starting point for a more robust > > fix. > > I tried (on Debian/bullseye with Emacs 29), but it doesn't seem to make > any difference here. With or without the patch, I get: > > larsi@xo:~/src/emacs/trunk$ emake; cat /tmp/script.el | ./src/emacs -Q --script /dev/stdin > Debugger entered--Lisp error: (file-missing "Cannot open load file" "No such file or directory" "/proc/162231/fd/pipe:[1065564]") > load("/proc/162231/fd/pipe:[1065564]" nil t t) > command-line-1(("-scriptload" "/dev/stdin")) > command-line() > normal-top-level() > > Perhaps something further has changed in the meantime? (You didn't post > an output of the error message you were getting...) I did post the complete output: the pipeline `cat script.el | emacs --no-init-file --no-site-file --script /dev/stdin` did not produce any output whatsoever for my reproducer. (Nothing to stdout, nothing to stderr.) Since the return-value from lseek was ignored prior to my patch, the failing seek did not necessarily lead to a diagnosed error at all — it could instead result in a spurious EOF on the next read, or a parse error from unintentionally continuing to read the file at the unmodified (pre-lseek) offset. I don't know what else may have regressed between when I mailed the patch and when you looked at it in Emacs 29, and unfortunately I don't have the time to look into it at the moment. ^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#48940: 27.2; regression: "emacs --script /dev/stdin" parses the script incorrectly when /dev/stdin is a pipe 2021-11-11 17:23 ` Bryan C. Mills via Bug reports for GNU Emacs, the Swiss army knife of text editors @ 2021-11-12 3:26 ` Lars Ingebrigtsen 0 siblings, 0 replies; 6+ messages in thread From: Lars Ingebrigtsen @ 2021-11-12 3:26 UTC (permalink / raw) To: Bryan C. Mills; +Cc: 48940 "Bryan C. Mills" <bcmills@google.com> writes: > I did post the complete output: the pipeline `cat script.el | emacs > --no-init-file --no-site-file --script /dev/stdin` did not produce any > output whatsoever for my reproducer. (Nothing to stdout, nothing to > stderr.) Ah, I see. I tracked down what's causing this new problem and fixed that, and after doing that, I can confirm that your patch fixes the problem with /dev/stdin-as-a-pipe, so I've pushed it to Emacs 29. -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2021-11-12 3:26 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2021-06-09 20:31 bug#48940: 27.2; regression: "emacs --script /dev/stdin" parses the script incorrectly when /dev/stdin is a pipe Bryan C. Mills via Bug reports for GNU Emacs, the Swiss army knife of text editors 2021-06-13 10:26 ` Eli Zaretskii 2021-06-14 21:37 ` Bryan C. Mills via Bug reports for GNU Emacs, the Swiss army knife of text editors 2021-11-11 5:09 ` Lars Ingebrigtsen 2021-11-11 17:23 ` Bryan C. Mills via Bug reports for GNU Emacs, the Swiss army knife of text editors 2021-11-12 3:26 ` Lars Ingebrigtsen
Code repositories for project(s) associated with this external index https://git.savannah.gnu.org/cgit/emacs.git https://git.savannah.gnu.org/cgit/emacs/org-mode.git This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.