bug#34174: 27.0.50; Calling `par' via shell-command-on-region always errors out

bug#34174: 27.0.50; Calling `par' via shell-command-on-region always errors out

[Kaushal Modi]

[-- Attachment #1: Type: text/plain, Size: 4820 bytes --]

Hello,

I have found a regression in the behavior of shell-command-on-region,
though specifically for a third party binary `par'. par can be obtained
from http://www.nicemice.net/par/.

How to reproduce the error:

1. Install par and put it in PATH (sorry, I cannot reproduce this error
with any other utility).
2. Open emacs -Q.
3. Select the two lines that show up by default in the scratch buffer: ";;
This buffer is for text that is not saved, and for Lisp evaluation.
;; To create a file, visit it with C-x C-f and enter text in its buffer.
".
4. Call `M-| par'.

You will get this error on emacs master emacs -Q , but *not* on emacs 26.1:

===
ERROR in parse_header: buffer was ';; This buffer is for te'
error: client: reader got error Success
===

What's surprising is that I have selected the whole 2 lines to be sent to
par, but the error from par says:

    buffer was ';; This buffer is for te'

I cannot reproduce this error on 26.1.


I looked at the git revision history that affected shell-command-on-region
that affected in recent times, but I couldn't find any.

But I do see
https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=d1ec3a0a8e4d7d56ebc1e4fa743130b9974ac6a8
affecting callproc.c/call-process-region.

Can you please review if that change could have caused this regression?

Thanks.

In GNU Emacs 27.0.50 (build 11, x86_64-pc-linux-gnu, GTK+ Version 2.24.23)
 of 2019-01-22
Repository revision: 9eaf5e052a802a7b0560f983f2bb786e13ce2488
Repository branch: master
Windowing system distributor 'Open Text', version 11.0.11505
System Description: Red Hat Enterprise Linux Workstation release 6.8
(Santiago)

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Mark set
ERROR in parse_header: buffer was ';; This buffer is for te'
error: client: reader got error Success

Configured using:
 'configure --with-modules
 --prefix=/home/kmodi/usr_local/apps/6/emacs/master
 '--program-transform-name=s/^ctags$/ctags_emacs/' --with-harfbuzz
 --with-dumping=unexec 'CPPFLAGS=-I/home/kmodi/stowed/include
 -I/home/kmodi/usr_local/6/include -I/usr/include/freetype2
 -I/usr/include' 'CFLAGS=-O2 -march=native'
 'LDFLAGS=-L/home/kmodi/stowed/lib -L/home/kmodi/stowed/lib64
 -L/home/kmodi/usr_local/6/lib -L/home/kmodi/usr_local/6/lib64''

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GSETTINGS GLIB
NOTIFY INOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF
XFT ZLIB TOOLKIT_SCROLL_BARS GTK2 X11 XDBE XIM MODULES THREADS GMP

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  shell-dirtrack-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny seq byte-opt gv
bytecomp byte-compile cconv dired dired-loaddefs format-spec rfc822 mml
easymenu mml-sec password-cache epa derived epg epg-config gnus-util
rmail rmail-loaddefs time-date mm-decode mm-bodies mm-encode mail-parse
rfc2231 mailabbrev gmm-utils mailheader cl-loaddefs cl-lib sendmail
rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils shell pcomplete
comint ansi-color ring elec-pair mule-util tooltip eldoc electric
uniquify ediff-hook vc-hooks lisp-float-type mwheel term/x-win x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page menu-bar rfn-eshadow isearch timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite charscript charprop case-table epa-hook jka-cmpr-hook
help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote threads dbusbind inotify dynamic-setting system-font-setting
font-render-setting move-toolbar gtk x-toolkit x multi-tty
make-network-process emacs)

Memory information:
((conses 16 101350 8692)
 (symbols 48 20712 1)
 (strings 32 30382 2230)
 (string-bytes 1 841942)
 (vectors 16 15780)
 (vector-slots 8 517527 12352)
 (floats 8 43 74)
 (intervals 56 255 0)
 (buffers 992 12)
 (heap 1024 25874 879))


--
Kaushal Modi

[-- Attachment #2: Type: text/html, Size: 6060 bytes --]

bug#34174: 27.0.50; Calling `par' via shell-command-on-region always errors out

[Paul Eggert]

To help debug the problem, try running 'truss -o /tmp/par.tr par' 
instead of 'par', under both the old and new Emacs, and spot the difference.

Try also running 'truss -o /tmp/emacs.tr emacs -Q' instead of plain 'emacs'.

bug#34174: 27.0.50; Calling `par' via shell-command-on-region always errors out

[Kaushal Modi]

[-- Attachment #1: Type: text/plain, Size: 864 bytes --]

On Wed, Jan 23, 2019 at 12:30 PM Paul Eggert <eggert@cs.ucla.edu> wrote:

> To help debug the problem, try running 'truss -o /tmp/par.tr par'
> instead of 'par', under both the old and new Emacs, and spot the
> difference.
>
> Try also running 'truss -o /tmp/emacs.tr emacs -Q' instead of plain
> 'emacs'.
>

I don't have truss installed. But I googled about it, and looks like strace
is equivalent.

I ran:

strace -o /tmp/par.tr par, and
strace -o /tmp/emacs.tr emacs -Q

Looking at the .tr files, they could possibly contain work sensitive data.
So I am not sure if I should share those files publicly.

What should I be looking for in there?

Also, as I am playing with this more, I realize that I cannot 100%
reproduce the problem on emacs -Q on master branch.

[ So the problem, as you guessed, seems to be with my OS (RHEL 6.8) .. But
why only `par`? hmm ]

[-- Attachment #2: Type: text/html, Size: 1587 bytes --]

bug#34174: 27.0.50; Calling `par' via shell-command-on-region always errors out

[Paul Eggert]

On 1/23/19 9:37 AM, Kaushal Modi wrote:
> I ran:
>
> strace -o /tmp/par.tr <http://par.tr> par, and
> strace -o /tmp/emacs.tr <http://emacs.tr> emacs -Q
>
> Looking at the .tr files, they could possibly contain work sensitive 
> data. So I am not sure if I should share those files publicly.

We care about the I/O requests involving the text you're reformatting, 
so I suggest running 'par' on some innocuous text and just looking at 
the associated syscalls (assuming you can reproduce the problem).

Also, please try running the shell command 'fmt' on the same text, to 
see if that can reproduce the problem. If it doesn't reproduce the 
problem, see how fmt's system calls differ from par's. I mention 'fmt' 
because it's similar to 'par' and I can easily run 'fmt' here.


> [ So the problem, as you guessed, seems to be with my OS (RHEL 6.8) .. 
> But why only `par`? hmm ]
Wow, that's older than even the most-ancient RHEL host I have ready 
access to (RHEL 6.10). I could not reproduce the problem on that host, 
using 'fmt'.

bug#34174: 27.0.50; Calling `par' via shell-command-on-region always errors out

[Kaushal Modi]

[-- Attachment #1: Type: text/plain, Size: 1188 bytes --]

On Wed, Jan 23, 2019 at 1:05 PM Paul Eggert <eggert@cs.ucla.edu> wrote:

> On 1/23/19 9:37 AM, Kaushal Modi wrote:
> > I ran:
> >
> > strace -o /tmp/par.tr <http://par.tr> par, and
> > strace -o /tmp/emacs.tr <http://emacs.tr> emacs -Q
> >
> > Looking at the .tr files, they could possibly contain work sensitive
> > data. So I am not sure if I should share those files publicly.
>
> We care about the I/O requests involving the text you're reformatting,
> so I suggest running 'par' on some innocuous text and just looking at
> the associated syscalls (assuming you can reproduce the problem).
>
> Also, please try running the shell command 'fmt' on the same text, to
> see if that can reproduce the problem. If it doesn't reproduce the
> problem, see how fmt's system calls differ from par's. I mention 'fmt'
> because it's similar to 'par' and I can easily run 'fmt' here.
>


Hi Paul,

Thanks for your time. I hate it when this happens.

This issue got automagically "solved". I cannot recreate this issue any
more.
All I did was rebuild Emacs to the latest master.

[ As I did not understand the root cause of this issue, I am certain that
this will repeat at some point. ]

Kaushal

[-- Attachment #2: Type: text/html, Size: 2050 bytes --]