From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Spencer Baugh Newsgroups: gmane.emacs.devel Subject: Re: Native OS pipelines in eshell and Emacs Date: Tue, 28 May 2024 21:43:54 -0400 Message-ID: References: <08f95b98-7338-4365-8b55-7c337e57d447@gutov.dev> Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="000000000000a004bd06198de2df" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="18433"; mail-complaints-to="usenet@ciao.gmane.io" Cc: emacs-devel@gnu.org, johnw@gnu.org, spwhitton@spwhitton.name To: Dmitry Gutov Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Wed May 29 04:23:05 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1sC8y0-0004Zv-F8 for ged-emacs-devel@m.gmane-mx.org; Wed, 29 May 2024 04:23:04 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sC8wx-0004nR-9I; Tue, 28 May 2024 22:21:59 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sC8MJ-0006ey-I5 for emacs-devel@gnu.org; Tue, 28 May 2024 21:44:07 -0400 Original-Received: from mxout5.mail.janestreet.com ([64.215.233.18]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sC8MH-0003KI-Sb for emacs-devel@gnu.org; Tue, 28 May 2024 21:44:07 -0400 Original-Received: from mail-ej1-f70.google.com ([209.85.218.70]) by mxgoog2.mail.janestreet.com with esmtps (TLS1.3:TLS_AES_128_GCM_SHA256:128) (Exim 4.97.1) id 1sC8MG-00000007qkT-3etU for emacs-devel@gnu.org; Tue, 28 May 2024 21:44:04 -0400 Original-Received: by mail-ej1-f70.google.com with SMTP id a640c23a62f3a-a6349bf258bso61983966b.1 for ; Tue, 28 May 2024 18:44:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=janestreet.com; s=google; t=1716947044; x=1717551844; darn=gnu.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=+/19l1jFS5roBmXRL1eimO8FIl8UyjvCaPGNfKJ98o8=; b=lQmk6/NMJAAjsNQIbZ73VZnJCdp4EqDO8FeJX+1J6n6IazAYmfgbLhJhDy/iRv1KAx ojzDdKG9Ic2I/xDeURkfzyfdYdtdgWBQRe2k5HSIn+fff1pdLrRRzzv9R/CfEa7Ng06K N56DOUr0O89D0GiWLiajg+qSU7UOGXcy//xRA= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=janestreet.com; s=waixah; t=1716947044; bh=+/19l1jFS5roBmXRL1eimO8FIl8UyjvCaPGNfKJ98o8=; h=References:In-Reply-To:From:Date:Subject:To:Cc; b=mrf+0DZJ7HYb6msNIUSSTq4QvZ/YTNR+zoaY6bEQvAu+06IbIFUemWm6AWsBL1dxa EFV534UH9R02KWl/XmDixkti+bbZp7OlEITd7B2WWiApueN80GsZC5juKHAtDcozPv +CaJXiyNE7aLScTNAoociPEL/8AasQWQMNxzIA8NBN4eoO7SU1j4ojNnqawyc7Uk26 f6TNfTOwpjERFApoS2rYjlEmCmeqNmb/4mk8WnGowfBEnjS2iwbCvUUyO40x94rhLj 9LukdK7cGr+CxPH3vKjk3hA518sFUFevCiXvXschLC0/cOW8GjDOy80FE54qBNQjxY etw4R16SzQH7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716947044; x=1717551844; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=+/19l1jFS5roBmXRL1eimO8FIl8UyjvCaPGNfKJ98o8=; b=MPUJvyyqe8eQAwh7buYMgtgqq1NnpbOY3+b0sUZzQ/OostBgPMPQXyIHuPEiMtRxoe cCxgDYX0h9UOlXatT5tYhAzD7lSIeDvW3cOOGW1NMQVoc2olFmXTXKMBsMgBmLjgtvzm 4v1NSRMQvbKeoT6hslCoH1vdzXkktuW7Nak1NX+oKMpgyR7sh3UPvtb1r1Z8aAi0p4EB wouf56i064vPMr3qlPnUdLXGgW9Eol7X+OAS94TJvkfqTfOWT7yQMdTCj13vgMwETTv+ pLetubeu9TQj/wSarG9JSgKbZBRUaZHY7olv6ci4dR77ah/oOFK15aBIpp50TCMDY8By bB6A== X-Gm-Message-State: AOJu0Yz9gzndwg7h89hzZaJJBuYpw2jmTu9aY8Q8n+LWTwuaPiizQAbZ TwLuxQcoUoFNFf4R5LPQJp9ODj5pcvQgHbOioaWPdt2ZYtmeGGXjAfU3kJLTeIlTTi+ov3yoaiv J8Z4rAOrsDpKCQXSeK19XhHrIHhj1b+1ge5zMJ76EpLJxPpryI+lnFJqIP4c/m1Qk08dV61t3kI L+SuGVooObBi8OcJMvksDs6vY= X-Received: by 2002:a17:906:1982:b0:a62:a63c:1900 with SMTP id a640c23a62f3a-a62a63c1a9fmr721143266b.5.1716947043829; Tue, 28 May 2024 18:44:03 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHhILeiViCHkAVLz3PY7TanQSEeGn9CPR9A7zR9trsaxisclskrDY4znzsF6O69hHwd9rlmIw49VmqN8g7YBS8= X-Received: by 2002:a17:906:1982:b0:a62:a63c:1900 with SMTP id a640c23a62f3a-a62a63c1a9fmr721142566b.5.1716947043478; Tue, 28 May 2024 18:44:03 -0700 (PDT) In-Reply-To: <08f95b98-7338-4365-8b55-7c337e57d447@gutov.dev> Received-SPF: pass client-ip=64.215.233.18; envelope-from=sbaugh@janestreet.com; helo=mxout5.mail.janestreet.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Tue, 28 May 2024 22:21:58 -0400 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:319667 Archived-At: --000000000000a004bd06198de2df Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Tue, May 28, 2024, 9:21=E2=80=AFPM Dmitry Gutov wrote= : > > Specifically, the new feature would be something like an :stdin argumen= t > > to make-process which allows a make-pipe-process (or other process) to > > be passed as stdin, and grabs the output file descriptor from that > > process (what Emacs would normally read) and passes it down as stdin fo= r > > the new process instead. > > It would be doubly interesting if we manage to implement it so that > Tramp would be able to connect two processes directly without > round-tripping the i/o from the remote host to local and back to remote. > That's a major source of latency in project-find-regexp on remote. > Unfortunately this is almost impossibly hard. But, I actually have worked extensively on doing this specific impossible thing (remote process APIs that are powerful enough to do this) so I will eventually try to implement them for Emacs and TRAMP. It would allow full make-process support in TRAMP as well as a make-pipe-process which represents a pipe existing on a remote system. Anyway, in the short term it will probably only work efficiently for local processes, with remote project-files having to roundtrip through the local Emacs. --000000000000a004bd06198de2df Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Tue, May 28, 2024, 9:21=E2=80=AFPM Dmitry Gutov <= ;dmitry@gutov.dev> wrote:
<= blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px= #ccc solid;padding-left:1ex"> > Specifically, the new feature would be something like an :stdin argume= nt
> to make-process which allows a make-pipe-process (or other process) to=
> be passed as stdin, and grabs the output file descriptor from that
> process (what Emacs would normally read) and passes it down as stdin f= or
> the new process instead.

It would be doubly interesting if we manage to implement it so that
Tramp would be able to connect two processes directly without
round-tripping the i/o from the remote host to local and back to remote. That's a major source of latency in project-find-regexp on remote.
<= /blockquote>

Unfor= tunately this is almost impossibly hard.=C2=A0 But, I actually have worked = extensively on doing this specific impossible thing (remote process APIs th= at are powerful enough to do this) so I will eventually try to implement th= em for Emacs and TRAMP.=C2=A0 It would allow full make-process support in T= RAMP as well as a make-pipe-process which represents a pipe existing on a r= emote system.

Anyway, in= the short term it will probably only work efficiently for local processes,= with remote project-files having to roundtrip through the local Emacs.

=
--000000000000a004bd06198de2df--