bug#48129: 28.0.50; Per file-handle (stdin, stdout and stderr) process-connection-type variable

bug#48129: 28.0.50; Per file-handle (stdin, stdout and stderr) process-connection-type variable

[Thomas Hisch]

I would like that the configuration options of the three file
handles (stdin, stdout and stderr), when a new process is created
in emacs, are improved.  What I'm specifically asking for is to
add support for individually specifying for each file-handle
whether it should be connected to a pty or not.  Currently the
value of the process-connection-type variable applies to all
three file handles at once, which leads to a problem with the
following use-case:

Usually a program outputs ANSI escape sequences when its stdout
file-handle is connected to a pty. It is currently not possible to
communicate with a subprocess that is created by `start-file-process' by
writing to a pty (via stdin) and avoiding that this subprocess writes
ANSI escape sequences to stdout. This use-case is needed by magit, which
currently doesn't have support for displaying ANSI escape sequences but
it needs to communicate with it's started subprocesses via a PTY
(https://github.com/magit/magit/issues/3549).

The reason why I need this feature is that ANSI escape sequences may be
output by git-hooks (scripts stored in .git/hooks) if git is run from a
terminal/pty. Magit uses (setq process-connection-type t) when creating
subprocesses s.t. it can commuicate via the stdin file-handle that is
connected to a pty with the subprocess.

Here is an small example that shows that all file handles can
either be connected to a pty (process-connection-type t) or none
of them (process-connection-type nil).

(let ((process-connection-type nil)
      (process-buf (generate-new-buffer "*temp*")))
    (apply #'start-file-process
           "isatty"
           process-buf
           "/tmp/output-isatty.py" nil))

The output-isatty.pt looks like

#!/usr/bin/env python3
import sys
print(f'in: {sys.stdin.isatty()}')
print(f'out: {sys.stdout.isatty()}')
print(f'err: {sys.stderr.isatty()}')


If the above elisp code is run

in: False
out: False
err: False

is written to the *temp* buffer if process-connection-type is nil.
And

in: True
out: True
err: True

is written to the *temp* buffer if process-connection-type is t.

In the 'subprocess' and the 'pty' modules in the python stdlib the above
use-case is already supported (see the possible values for the
stdin,stdout and stderr params). Here is a small demo that attaches a
pty to stdin and two normal pipes to stdout and stderr:

import os
import pty
import subprocess

master, slave = pty.openpty()

# see https://docs.python.org/3.5/library/subprocess.html#popen-constructor
proc = subprocess.Popen(
    ('./t.py',),
    stdin=slave,
    stdout=subprocess.PIPE,
    stderr=subprocess.PIPE,
)

os.close(slave)
with os.fdopen(master, 'wb', 0) as stdin:
    stdin.write(b'hello\n')
    out, err = proc.communicate()

print('RETC:')
print(proc.returncode)
print('OUT:')
print(out.decode(), end='')
print('ERR:')
print(err.decode(), end='')



In GNU Emacs 28.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.28, cairo version 1.16.0)
 of 2021-04-29 built on NBPF1PQX4B
Repository revision: 0e8c8628854d786e2f99dd2ac35ace88a5ee7924
Repository branch: master
Windowing system distributor 'Fedora Project', version 11.0.12011000
System Description: Fedora 33 (Workstation Edition)

Configured using:
 'configure 'CFLAGS=-march=native -O3' --prefix=/opt/emacs --with-pgtk
 --with-xpm --with-jpeg --with-gif --with-tiff --with-modules
 --with-native-compilation --with-json'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
JSON LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES NATIVE_COMP
NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP SOUND THREADS TIFF
TOOLKIT_SCROLL_BARS X11 XDBE XIM XPM GTK3 ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: ELisp/d

Minor modes in effect:
  persp-mode: t
  counsel-projectile-mode: t
  counsel-mode: t
  ivy-rich-mode: t
  ivy-mode: t
  global-ethan-wspace-mode: t
  ethan-wspace-mode: t
  ethan-wspace-clean-many-nls-eof-mode: t
  ethan-wspace-clean-no-nl-eof-mode: t
  ethan-wspace-clean-eol-mode: t
  ethan-wspace-clean-tabs-mode: t
  save-place-mode: t
  show-paren-mode: t
  delete-selection-mode: t
  global-hi-lock-mode: t
  hi-lock-mode: t
  winner-mode: t
  winum-mode: t
  global-undo-tree-mode: t
  undo-tree-mode: t
  thi::spell-fu-mode: t
  spell-fu-mode: t
  savehist-mode: t
  rainbow-delimiters-mode: t
  idle-highlight-mode: t
  async-bytecomp-package-mode: t
  highlight-function-calls-mode: t
  recentf-mode: t
  global-git-commit-mode: t
  magit-auto-revert-mode: t
  global-flycheck-mode: t
  flycheck-mode: t
  flx-ido-mode: t
  evil-mode: t
  evil-local-mode: t
  envrc-global-mode: t
  envrc-mode: t
  yas-minor-mode: t
  shell-dirtrack-mode: t
  projectile-mode: t
  outline-minor-mode: t
  override-global-mode: t
  bug-reference-prog-mode: t
  url-handler-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  global-prettify-symbols-mode: t
  prettify-symbols-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  size-indication-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t
  hs-minor-mode: t

Load-path shadows:
/home/thomas.hisch/.emacs.d/user-lisp/custom hides /opt/emacs/share/emacs/28.0.50/lisp/custom

Features:
(shadow sort mail-extr emacsbug sendmail shortdoc cl-print url-cache
rng-xsd xsd-regexp rng-cmpct rng-nxml rng-valid rng-loc rng-uri
rng-parse nxml-parse rng-match rng-dt rng-util rng-pttrn nxml-ns
nxml-mode nxml-outln nxml-rap sgml-mode facemenu nxml-util nxml-enc
xmltok vc-mtn vc-hg vc-bzr vc-src vc-sccs vc-svn vc-cvs vc-rcs
hippie-exp magit-extras origami origami-parsers origami-util yapfify
python pulse tramp-archive tramp-gvfs zeroconf deadgrep perspective
org-element avl-tree generator ol-eww eww url-queue mm-url ol-rmail
ol-mhe ol-irc ol-info ol-gnus nnselect gnus-search eieio-opt speedbar
ezimage dframe gnus-art mm-uu mml2015 mm-view mml-smime smime dig
gnus-sum shr kinsoku svg dom gnus-group gnus-undo gnus-start gnus-dbus
dbus xml gnus-cloud nnimap nnmail mail-source utf7 netrc nnoo gnus-spec
gnus-int gnus-range gnus-win gnus nnheader ol-docview doc-view jka-compr
image-mode exif ol-bibtex bibtex ol-bbdb ol-w3m org ob ob-tangle ob-ref
ob-lob ob-table ob-exp org-macro org-footnote org-src ob-comint
org-pcomplete org-list org-faces org-entities org-version ob-emacs-lisp
ob-core ob-eval org-table ol org-keys org-compat org-macs org-loaddefs
cal-menu calendar cal-loaddefs vc-git misearch multi-isearch mule-util
magit-bookmark bookmark counsel-projectile counsel swiper ivy-rich
ivy-hydra ivy ivy-faces ivy-overlay colir
smart-mode-line-respectful-theme smart-mode-line-dark-theme
sanityinc-tomorrow-night-theme ethan-wspace hideshow url-tramp ims-jira
saveplace ffap paren delsel hi-lock thi-projects winner py-isort
format-all language-id reformatter winum which-key wgrep-ag vterm-toggle
tramp-sh vterm vterm-module term/xterm xterm visual-regexp
visual-fill-column undo-tree tree-sitter-langs tree-sitter-langs-build
tar-mode arc-mode archive-mode tree-sitter-hl tree-sitter
tree-sitter-load tree-sitter-cli tsc tsc-dyn tsc-dyn-get dired-aux
tsc-obsolete typescript-mode stickyfunc-enhance semantic/util-modes
semantic/util semantic semantic/tag semantic/lex semantic/fw mode-local
cedet ssh-config-mode sphinx-doc spell-fu smex smart-mode-line
rich-minority savehist rust-utils rust-mode rust-rustfmt rust-playpen
rust-compile rust-cargo rg rg-info-hack rg-menu rg-ibuffer rg-result
wgrep-rg wgrep rg-history rg-header cus-edit pp cus-load
rainbow-delimiters python-docstring pyimport shut-up protobuf-mode
paredit lsp-ui lsp-ui-flycheck lsp-ui-doc goto-addr lsp-ui-imenu
lsp-ui-peek lsp-ui-sideline face-remap lsp-ui-util dap-chrome dap-utils
kubernetes kubernetes-overview kubernetes-services kubernetes-secrets
kubernetes-namespaces kubernetes-jobs kubernetes-pod-line
kubernetes-ingress kubernetes-errors kubernetes-nodes
kubernetes-statefulsets kubernetes-deployments kubernetes-contexts
kubernetes-configmaps kubernetes-logs kubernetes-labels kubernetes-pods
kubernetes-yaml kubernetes-loading-container kubernetes-commands
kubernetes-utils kubernetes-timers kubernetes-kubectl kubernetes-process
term disp-table ehelp kubernetes-props kubernetes-popups
kubernetes-state kubernetes-vars kubernetes-modes kubernetes-ast
idle-highlight-mode helm-projectile helm-files helm-tags helm-buffers
helm-occur helm-grep helm-regexp helm-utils helm-locate helm-help
helm-types helm async-bytecomp helm-global-bindings helm-easymenu
helm-source helm-multi-match helm-lib async highlight-indentation
highlight-function-calls gerrit gerrit-rest recentf forge-list
forge-commands forge-semi forge-bitbucket buck forge-gogs gogs
forge-gitea gtea forge-gitlab glab forge-github ghub-graphql treepy
gsexp ghub gnutls forge-notify forge-revnote forge-pullreq forge-issue
forge-topic forge-post forge-repo forge forge-core forge-db closql
emacsql-sqlite emacsql emacsql-compiler url-http url-auth url-gw
magit-submodule magit-obsolete magit-popup magit-blame magit-stash
magit-reflog magit-bisect magit-push magit-pull magit-fetch magit-clone
magit-remote magit-commit magit-sequence magit-notes magit-worktree
magit-tag magit-merge magit-branch magit-reset magit-files magit-refs
magit-status magit magit-repos magit-apply magit-wip magit-log
which-func magit-diff smerge-mode diff diff-mode git-commit log-edit
message rfc822 mml mml-sec epa derived gnus-util rmail rmail-loaddefs
mailabbrev mail-utils gmm-utils mailheader pcvs-util add-log magit-core
magit-libgit libgit libegit2 magit-autorevert magit-margin
magit-transient magit-process with-editor server magit-mode transient
magit-git magit-utils crm flycheck-package package-lint let-alist finder
flycheck find-func flx-ido ido flx evil evil-keybindings
evil-integration evil-maps evil-commands reveal flyspell ispell
evil-jumps evil-command-window evil-types evil-search evil-ex
evil-macros evil-repeat evil-states evil-core advice evil-common
windmove calc calc-loaddefs calc-macs rect evil-digraphs evil-vars
equake envrc inheritenv ediff ediff-merg ediff-mult ediff-wind
ediff-diff ediff-help ediff-init ediff-util cc-mode cc-fonts cc-guess
cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs dumb-jump
popup dracula-theme dired dired-loaddefs doom-themes doom-themes-base
dap-mode dap-launch posframe dap-overlays lsp-mode lsp-protocol
yasnippet-snippets yasnippet tree-widget wid-edit spinner network-stream
puny nsm rmc inline imenu ht ewoc epg epg-config
color-theme-sanityinc-tomorrow cargo cargo-process docker-tramp
tramp-cache tramp tramp-loaddefs trampver tramp-integration files-x
tramp-compat shell pcomplete ls-lisp format-spec markdown-mode
edit-indirect color bufler projectile grep compile text-property-search
comint ansi-color ibuf-ext ibuffer ibuffer-loaddefs thingatpt
pretty-hydra hydra lv bufler-group-tree magit-section f s
dash-functional dash noutline outline vc vc-dispatcher diminish
autorevert filenotify quelpa-use-package parse-time iso8601 time-date
quelpa mm-decode mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045
mm-util ietf-drums mail-prsvr lisp-mnt help-fns radix-tree use-package
use-package-ensure use-package-delight use-package-diminish
use-package-bind-key bind-key use-package-core finder-inf xref project
ring bug-reference xdg tab-line pcase easy-mmode edmacro kmacro info
package browse-url url url-proxy url-privacy url-expand url-methods
url-history url-cookie url-domsuf url-util mailcap url-handlers
url-parse auth-source eieio eieio-core eieio-loaddefs password-cache
json map url-vars comp comp-cstr warnings subr-x rx cl-seq cl-macs
cl-extra help-mode seq byte-opt gv cl-loaddefs cl-lib bytecomp
byte-compile cconv iso-transl tooltip eldoc electric uniquify ediff-hook
vc-hooks lisp-float-type mwheel term/x-win x-win term/common-win x-dnd
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace
newcomment text-mode elisp-mode lisp-mode prog-mode register page
tab-bar menu-bar rfn-eshadow isearch easymenu timer select scroll-bar
mouse jit-lock font-lock syntax font-core term/tty-colors frame
minibuffer cl-generic cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite charscript charprop case-table epa-hook jka-cmpr-hook help
simple abbrev obarray cl-preloaded nadvice button loaddefs faces
cus-face macroexp files window text-properties overlay sha1 md5 base64
format env code-pages mule custom widget hashtable-print-readable
backquote threads dbusbind inotify dynamic-setting system-font-setting
font-render-setting cairo move-toolbar gtk x-toolkit x multi-tty
make-network-process nativecomp emacs)

Memory information:
((conses 16 2215206 1909118)
 (symbols 48 145665 937)
 (strings 32 583224 150267)
 (string-bytes 1 18958140)
 (vectors 16 200384)
 (vector-slots 8 6002760 1568162)
 (floats 8 1240 1330)
 (intervals 56 119338 63212)
 (buffers 992 87))
Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Mail ist nicht gestattet.

This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly prohibited.

IMS Nanofabrication GmbH FN: 208021 p, FB-Gericht: Wien; Sitz: Schreygasse 3, 1020, Vienna (Austria),Tel. +43 1 214 48 94; E-Mail: imsoffice@ims.co.at; DVR-Nr: 0374369; www.ims.co.at

bug#48129: 28.0.50; Per file-handle (stdin, stdout and stderr) process-connection-type variable

[Eli Zaretskii]

> From: Thomas Hisch <thomas.hisch@ims.co.at>
> Date: Sat, 1 May 2021 01:10:10 +0200
> 
> 
> I would like that the configuration options of the three file
> handles (stdin, stdout and stderr), when a new process is created
> in emacs, are improved.  What I'm specifically asking for is to
> add support for individually specifying for each file-handle
> whether it should be connected to a pty or not.  Currently the
> value of the process-connection-type variable applies to all
> three file handles at once, which leads to a problem with the
> following use-case:
> 
> Usually a program outputs ANSI escape sequences when its stdout
> file-handle is connected to a pty. It is currently not possible to
> communicate with a subprocess that is created by `start-file-process' by
> writing to a pty (via stdin) and avoiding that this subprocess writes
> ANSI escape sequences to stdout. This use-case is needed by magit, which
> currently doesn't have support for displaying ANSI escape sequences but
> it needs to communicate with it's started subprocesses via a PTY
> (https://github.com/magit/magit/issues/3549).

We have make-pipe-process; can't you solve your problem by connecting
your process's stdout to such a pipe-process?  If not, why not?

bug#48129: 28.0.50; Per file-handle (stdin, stdout and stderr) process-connection-type variable

[Thomas Hisch]

On 5/1/21 8:09 AM, Eli Zaretskii wrote:
>> From: Thomas Hisch <thomas.hisch@ims.co.at>
>> Date: Sat, 1 May 2021 01:10:10 +0200
>>
>>
>> I would like that the configuration options of the three file
>> handles (stdin, stdout and stderr), when a new process is created
>> in emacs, are improved.  What I'm specifically asking for is to
>> add support for individually specifying for each file-handle
>> whether it should be connected to a pty or not.  Currently the
>> value of the process-connection-type variable applies to all
>> three file handles at once, which leads to a problem with the
>> following use-case:
>>
>> Usually a program outputs ANSI escape sequences when its stdout
>> file-handle is connected to a pty. It is currently not possible to
>> communicate with a subprocess that is created by `start-file-process' by
>> writing to a pty (via stdin) and avoiding that this subprocess writes
>> ANSI escape sequences to stdout. This use-case is needed by magit, which
>> currently doesn't have support for displaying ANSI escape sequences but
>> it needs to communicate with it's started subprocesses via a PTY
>> (https://github.com/magit/magit/issues/3549).
>
> We have make-pipe-process; can't you solve your problem by connecting
> your process's stdout to such a pipe-process?  If not, why not?
>

Thx Eli for the fast reply! I had to study the docs of make-pipe-process
and make-process to understand what you mean. However, I'm not fully
sure if I fully get what you mean.

AFAICS the pipe process which is returned by make-pipe-process can be
passed to the :stderr keyword argument of make-process. Is that what you
head in mind?

According to the docs of make-process, process-connection-type is
ignored if :stderr is passed to make-process:

 >  This parameter and the value of ‘process-connection-type’ are
ignored if a non-‘nil’ value is specified for the ‘:stderr’ parameter;
in that case, the type will always be ‘pipe’.

What I want is that stdout and stderr are connected to a pipe but stdin
is connected to a pty.

Here is the new test script that I used

(let* (
       (stderr-buffer (get-buffer-create "*stderr*"))
       (stderr-proc (make-pipe-process :name "stderr"
                       :buffer stderr-buffer))
       (process-buf (get-buffer-create "*temp*")))
     (make-process
      :name "isatty"
      :buffer process-buf ;; it is not possible to pass the output of
make-pipe-process to :buffer
      :connection-type 'pty ;; unfortunately ignored if :stderr is used
      :stderr stderr-proc
      :command (list "/tmp/output-isatty.py")))

Best regards
Thomas
Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Mail ist nicht gestattet.

This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly prohibited.

IMS Nanofabrication GmbH FN: 208021 p, FB-Gericht: Wien; Sitz: Schreygasse 3, 1020, Vienna (Austria),Tel. +43 1 214 48 94; E-Mail: imsoffice@ims.co.at; DVR-Nr: 0374369; www.ims.co.at

bug#48129: 28.0.50; Per file-handle (stdin, stdout and stderr) process-connection-type variable

[Thomas Hisch]

On 5/2/21 12:19, Thomas Hisch wrote:
> On 5/1/21 8:09 AM, Eli Zaretskii wrote:
>>> From: Thomas Hisch <thomas.hisch@ims.co.at>
>>> Date: Sat, 1 May 2021 01:10:10 +0200
>>>
>>>
>>> I would like that the configuration options of the three file
>>> handles (stdin, stdout and stderr), when a new process is created
>>> in emacs, are improved.  What I'm specifically asking for is to
>>> add support for individually specifying for each file-handle
>>> whether it should be connected to a pty or not.  Currently the
>>> value of the process-connection-type variable applies to all
>>> three file handles at once, which leads to a problem with the
>>> following use-case:
>>>
>>> Usually a program outputs ANSI escape sequences when its stdout
>>> file-handle is connected to a pty. It is currently not possible to
>>> communicate with a subprocess that is created by `start-file-process' by
>>> writing to a pty (via stdin) and avoiding that this subprocess writes
>>> ANSI escape sequences to stdout. This use-case is needed by magit, which
>>> currently doesn't have support for displaying ANSI escape sequences but
>>> it needs to communicate with it's started subprocesses via a PTY
>>> (https://github.com/magit/magit/issues/3549).
>>
>> We have make-pipe-process; can't you solve your problem by connecting
>> your process's stdout to such a pipe-process?  If not, why not?
>>
>
> Thx Eli for the fast reply! I had to study the docs of make-pipe-process
> and make-process to understand what you mean. However, I'm not fully
> sure if I fully get what you mean.
>
> AFAICS the pipe process which is returned by make-pipe-process can be
> passed to the :stderr keyword argument of make-process. Is that what you
> head in mind?
>
> According to the docs of make-process, process-connection-type is
> ignored if :stderr is passed to make-process:
>
>>  This parameter and the value of ‘process-connection-type’ are
> ignored if a non-‘nil’ value is specified for the ‘:stderr’ parameter;
> in that case, the type will always be ‘pipe’.
>
> What I want is that stdout and stderr are connected to a pipe but stdin
> is connected to a pty.
>
> Here is the new test script that I used
>
> (let* (
>       (stderr-buffer (get-buffer-create "*stderr*"))
>       (stderr-proc (make-pipe-process :name "stderr"
>                       :buffer stderr-buffer))
>       (process-buf (get-buffer-create "*temp*")))
>     (make-process
>      :name "isatty"
>      :buffer process-buf ;; it is not possible to pass the output of
> make-pipe-process to :buffer
>      :connection-type 'pty ;; unfortunately ignored if :stderr is used
>      :stderr stderr-proc
>      :command (list "/tmp/output-isatty.py")))
>

Eli can you tell me if you meant that we should use make-process in
combination with make-pipe-process?

Best regards
Thomas
Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Mail ist nicht gestattet.

This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly prohibited.

IMS Nanofabrication GmbH FN: 208021 p, FB-Gericht: Wien; Sitz: Schreygasse 3, 1020, Vienna (Austria),Tel. +43 1 214 48 94; E-Mail: imsoffice@ims.co.at; DVR-Nr: 0374369; www.ims.co.at

bug#48129: 28.0.50; Per file-handle (stdin, stdout and stderr) process-connection-type variable

[Eli Zaretskii]

> Date: Fri, 7 May 2021 21:46:15 +0200
> From: Thomas Hisch <thomas.hisch@ims.co.at>
> CC: <48129@debbugs.gnu.org>
> 
> >> We have make-pipe-process; can't you solve your problem by connecting
> >> your process's stdout to such a pipe-process?  If not, why not?
> >>
> >
> > Thx Eli for the fast reply! I had to study the docs of make-pipe-process
> > and make-process to understand what you mean. However, I'm not fully
> > sure if I fully get what you mean.
> >
> > AFAICS the pipe process which is returned by make-pipe-process can be
> > passed to the :stderr keyword argument of make-process. Is that what you
> > head in mind?
> >
> > According to the docs of make-process, process-connection-type is
> > ignored if :stderr is passed to make-process:
> >
> >>  This parameter and the value of ‘process-connection-type’ are
> > ignored if a non-‘nil’ value is specified for the ‘:stderr’ parameter;
> > in that case, the type will always be ‘pipe’.
> >
> > What I want is that stdout and stderr are connected to a pipe but stdin
> > is connected to a pty.
> >
> > Here is the new test script that I used
> >
> > (let* (
> >       (stderr-buffer (get-buffer-create "*stderr*"))
> >       (stderr-proc (make-pipe-process :name "stderr"
> >                       :buffer stderr-buffer))
> >       (process-buf (get-buffer-create "*temp*")))
> >     (make-process
> >      :name "isatty"
> >      :buffer process-buf ;; it is not possible to pass the output of
> > make-pipe-process to :buffer
> >      :connection-type 'pty ;; unfortunately ignored if :stderr is used
> >      :stderr stderr-proc
> >      :command (list "/tmp/output-isatty.py")))
> >
> 
> Eli can you tell me if you meant that we should use make-process in
> combination with make-pipe-process?

If it's possible to solve your problem by using make-pipe-process, I'd
prefer such a solution.  Introducing yet another obscure option to
make-process should best be avoided, IMO.

bug#48129: 28.0.50; Per file-handle (stdin, stdout and stderr) process-connection-type variable

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> If it's possible to solve your problem by using make-pipe-process, I'd
> prefer such a solution.  Introducing yet another obscure option to
> make-process should best be avoided, IMO.

This was a month ago, and there was no followup, so I'm going to go
ahead and assume that the make-pipe-process workaround works (and that
seems like a pretty composable solution to me, too), and I'm closing
this bug report.

If there's more to be done here, please respond to the debbugs address
and we'll reopen.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#48129: 28.0.50; Per file-handle (stdin, stdout and stderr) process-connection-type variable

[Thomas Hisch]

Hi,

On 6/12/21 14:18, Lars Ingebrigtsen wrote:
> Eli Zaretskii <eliz@gnu.org> writes:
>
>> If it's possible to solve your problem by using make-pipe-process, I'd
>> prefer such a solution.  Introducing yet another obscure option to
>> make-process should best be avoided, IMO.

Yes I agree, if I could use make-pipe-process I would use it, but I
guess it doesn't work.


(let* ((stderr-buffer (get-buffer-create "*stderr*"))
        (stderr-pipe-proc (make-pipe-process :name "stderr"
                                             :buffer stderr-buffer))
        (stdout-buffer (get-buffer-create "*temp*")))
     (make-process
      :name "isatty"
      ;; what do I want?
      ;; I want stdin to be connected to a pty but not stdout
      :buffer stdout-buffer
      :connection-type 'pty
      :stderr stderr-pipe-proc
      :command (list "/tmp/output-isatty.py")))

What have I tried:

* toggling the connection-type arguement doesn't have any impact on the
output of the python script, which is run as a subprocess.

* if a stderr argument is passed to make-process and the value is a the
pipe-process, then both stdin as well as stdout in the subprocess are
connected non-pty devices

* if a stderr argument is not passed to make-process, then both stdin as
well as stdout in the subprocess are connected to pty devices.


Can you tell me if I should use make-process/make-pipe-process in a
different way?


> This was a month ago, and there was no followup, so I'm going to go
> ahead and assume that the make-pipe-process workaround works (and that
> seems like a pretty composable solution to me, too), and I'm closing
> this bug report.
>
> If there's more to be done here, please respond to the debbugs address
> and we'll reopen.
>

sry for the late reply. Unfortunately make-pike-process doesn't work for
me, or at least I don't know how I should use it.



Best regards,
Thomas




Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Mail ist nicht gestattet.

This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly prohibited.

IMS Nanofabrication GmbH FN: 208021 p, FB-Gericht: Wien; Sitz: Schreygasse 3, 1020, Vienna (Austria),Tel. +43 1 214 48 94; E-Mail: imsoffice@ims.co.at; DVR-Nr: 0374369; www.ims.co.at

bug#48129: 28.0.50; Per file-handle (stdin, stdout and stderr) process-connection-type variable

[Eli Zaretskii]

> Date: Sat, 12 Jun 2021 16:23:29 +0200
> CC: <48129@debbugs.gnu.org>
> From: Thomas Hisch <thomas.hisch@ims.co.at>
> 
> > Eli Zaretskii <eliz@gnu.org> writes:
> >
> >> If it's possible to solve your problem by using make-pipe-process, I'd
> >> prefer such a solution.  Introducing yet another obscure option to
> >> make-process should best be avoided, IMO.
> 
> Yes I agree, if I could use make-pipe-process I would use it, but I
> guess it doesn't work.
> 
> 
> (let* ((stderr-buffer (get-buffer-create "*stderr*"))
>         (stderr-pipe-proc (make-pipe-process :name "stderr"
>                                              :buffer stderr-buffer))
>         (stdout-buffer (get-buffer-create "*temp*")))
>      (make-process
>       :name "isatty"
>       ;; what do I want?
>       ;; I want stdin to be connected to a pty but not stdout
>       :buffer stdout-buffer
>       :connection-type 'pty
>       :stderr stderr-pipe-proc
>       :command (list "/tmp/output-isatty.py")))
> 
> What have I tried:
> 
> * toggling the connection-type arguement doesn't have any impact on the
> output of the python script, which is run as a subprocess.
> 
> * if a stderr argument is passed to make-process and the value is a the
> pipe-process, then both stdin as well as stdout in the subprocess are
> connected non-pty devices
> 
> * if a stderr argument is not passed to make-process, then both stdin as
> well as stdout in the subprocess are connected to pty devices.

Indeed, I forgot that using :stderr with make-process automatically
resets the process connection typ to use pipes for all the 3 standard
handles.  I guess there's some technical issue underlying that?
Because we had that restriction since day one.  Perhaps using PTYs
means that stderr must be redirected to the same device as stdout or
something?

bug#48129: 28.0.50; Per file-handle (stdin, stdout and stderr) process-connection-type variable

[Lars Ingebrigtsen]

Thomas Hisch <thomas.hisch@ims.co.at> writes:

> sry for the late reply. Unfortunately make-pike-process doesn't work for
> me, or at least I don't know how I should use it.

OK; reopening.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#48129: 28.0.50; Per file-handle (stdin, stdout and stderr) process-connection-type variable

[Thomas Hisch]

On 6/12/21 17:31, Eli Zaretskii wrote:
>> * if a stderr argument is not passed to make-process, then both stdin as
>> well as stdout in the subprocess are connected to pty devices.
>
> Indeed, I forgot that using :stderr with make-process automatically
> resets the process connection typ to use pipes for all the 3 standard
> handles.  I guess there's some technical issue underlying that?

I guess you mean a technical issue in emacs and not in the OS, right?
cpython e.g. doesn't have this limitation. In cpython you can pass
one of the following values to each file handle (stdin, stdout, stderr)
  in the subprocess module:

(see https://docs.python.org/3.10/library/subprocess.html#subprocess.Popen)

subprocess.DEVNULL

     Special value that can be used as the stdin, stdout or stderr
argument to Popen and indicates that the special file os.devnull will be
used.

subprocess.PIPE

     Special value that can be used as the stdin, stdout or stderr
argument to Popen and indicates that a pipe to the standard stream
should be opened. Most useful with Popen.communicate().

subprocess.STDOUT

     Special value that can be used as the stderr argument to Popen and
indicates that standard error should go into the same handle as standard
output.


It is also possible to use

 > an existing file descriptor (a positive integer), an existing file
object, and None.

If you want to assign a file handle to pty you have to pass a file
descriptor to one of the handles (stdin,stdout,stderr)


It would be nice if emacs were as flexible as cpython w.r.t. configuring
file handles for subprocesses.


> Because we had that restriction since day one.  Perhaps using PTYs
> means that stderr must be redirected to the same device as stdout or
> something?
>
Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Mail ist nicht gestattet.

This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly prohibited.

IMS Nanofabrication GmbH FN: 208021 p, FB-Gericht: Wien; Sitz: Schreygasse 3, 1020, Vienna (Austria),Tel. +43 1 214 48 94; E-Mail: imsoffice@ims.co.at; DVR-Nr: 0374369; www.ims.co.at