From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Jim Porter Newsgroups: gmane.emacs.devel Subject: Re: Native OS pipelines in eshell and Emacs Date: Tue, 28 May 2024 12:56:27 -0700 Message-ID: References: <85a89224-b032-b083-9825-2ae215ae6301@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="32236"; mail-complaints-to="usenet@ciao.gmane.io" To: Spencer Baugh , emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Tue May 28 21:57:31 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1sC2ws-0008Av-5j for ged-emacs-devel@m.gmane-mx.org; Tue, 28 May 2024 21:57:30 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sC2w1-0000JS-1K; Tue, 28 May 2024 15:56:37 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sC2vy-0000Im-8Z for emacs-devel@gnu.org; Tue, 28 May 2024 15:56:34 -0400 Original-Received: from mail-pl1-x630.google.com ([2607:f8b0:4864:20::630]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sC2vw-0007Iz-29 for emacs-devel@gnu.org; Tue, 28 May 2024 15:56:33 -0400 Original-Received: by mail-pl1-x630.google.com with SMTP id d9443c01a7336-1f4a6a54416so1417555ad.0 for ; Tue, 28 May 2024 12:56:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1716926189; x=1717530989; darn=gnu.org; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:mime-version:date:message-id:from:to:cc :subject:date:message-id:reply-to; bh=SBiEmYDJEu+cZMfSE9Kuk3clGqljfA9VV0MqGSHeWow=; b=Wz+10sOYbXhiSY4biTdPULPbi5gmgflPGUfM07UHve0HKx/Xcviu7NKo0UqafHsZUU HK4G3qsn9vg4bmP6DSMWrKgs0aR8zXWq1ZMWkNL9KTin8LDrZmXW1FdOjhv6Z4nbUj8W bwf8ZjJuK4i6FIZtkf2GfftnnXA/FK04Lhzx963HzkNb6Msw2thE9mJhZJHdUESRbGuH qBZsqq4bFKJsBsMvsSV0xoTF9UcNFPA9+Dg44ez1YJ3tphl0aVx624pDlpmzq/g6mgjI PfeLABh7KoAwUgA63hXL0IuT0kgpYa6LfHgMBdD9s6weJzp3bnjleANBGGWjg+kjX4YL Mnbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716926189; x=1717530989; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=SBiEmYDJEu+cZMfSE9Kuk3clGqljfA9VV0MqGSHeWow=; b=vCtuFc68HsOCcaUJGY7zrYE2kGYDSVheLn7bIISZXGFU0/wIDVJwwtsu4pjJ5tQ2+1 dWAOuM6hoQEKQDCMXBPx1KmeL1nyhoOUsD68CkevdojW3/YzcTi+cg0iE4yDAJwsro1i QxvtB5hohQkVXfoJSJjyTY4GLoYnKFaDhs9EY4rLsVvuJvscOyoL/nVnzX3U6wfnCEi/ 11bpjyZ9yX6446FQ9q94pTSVwl/za0lLwmLeklbvjbCc7GpNrF84JhyDJzz4ELS7bseN XCL+wuQc7Co7pf/sIv33hsOXtJiwgRgZ9JE8v/d2BlCfYXdj7+7uAIKV6NRZ3AU9W/Sh O7pA== X-Forwarded-Encrypted: i=1; AJvYcCVFhNv94kmrlhUj79XRGTQreQUCczh7/E8QGAF7y9VbYSCjyujMa0JJDmJob4v3RTJdq7jIdQZC8z0GdoUzBIu0Da9N X-Gm-Message-State: AOJu0Yy8bACMeRKJJ+oASQFsgzuHbG4ckhmR1bFAi1x3PrCBOe5tVSjr t5vSAbpHd/4JEBh/7BjDkJFC6o/H+UQDWgRklIdmaYbRLRY/Z9m2 X-Google-Smtp-Source: AGHT+IHB0WZ7OLCO3nu377HPZWagcXcEgyrg9YvSR34l7w8vzhTM0/gGiVcHtKoWDGb1xI1eYFoj7g== X-Received: by 2002:a17:902:f551:b0:1f0:8cbf:c1b5 with SMTP id d9443c01a7336-1f4ea9942a7mr984565ad.16.1716926188829; Tue, 28 May 2024 12:56:28 -0700 (PDT) Original-Received: from [192.168.1.2] (syn-023-240-098-037.res.spectrum.com. [23.240.98.37]) by smtp.googlemail.com with ESMTPSA id d9443c01a7336-1f4960ca3f6sm45152475ad.164.2024.05.28.12.56.28 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 28 May 2024 12:56:28 -0700 (PDT) Content-Language: en-US In-Reply-To: Received-SPF: pass client-ip=2607:f8b0:4864:20::630; envelope-from=jporterbugs@gmail.com; helo=mail-pl1-x630.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:319657 Archived-At: On 5/28/2024 11:38 AM, Spencer Baugh wrote: > Jim Porter writes: >> I've worked on this previously, and even put together a hacky sketch >> of how it would work before abandoning it due to a bunch of >> complexities in Eshell that make this infeasible (in my opinion, >> anyway). As the current Eshell maintainer, I'd (softly) suggest you >> turn back now, unless you're willing to go down a fairly deep rabbit >> hole. > > That is fair, but since supporting :stdin in Emacs would be useful for > project.el anyway, I'm motivated to do this even if eshell won't > immediately benefit. > > So I'm especially interested in anything you have to share about the C > side of this, if anything. Don't let me discourage you from the C side of things. :) I think it would probably be pretty useful overall. As for advice on the C implementation, I don't have anything much in particular. I *think* this should be reasonably straightforward. The one wrinkle I got a bit bogged down in was with 'make-pipe-process'. That really creates 2 pipes under the hood, and for various reasons, I wanted to be able to use a single OS pipe from that, where I could (depending on the specific task) either write into it (pushing the data into stdin of a child process) or read out of it (pulling the data out of a child process's stdout). The above would probably make it easier for Eshell, since my plan was to use 'make-pipe-process' and hook it up into the Eshell IO code. (See 'eshell-get-target' and friends.) > Ah, those are indeed real and annoying concerns. > > But if extpipe is able to mandate this, doesn't that mean there is some > way to get this information? That is, we're able to detect "all the > commands are external programs" and "we're running on the local host". Well, extpipe does this by forcing the user to knowingly opt into a totally different behavior (just one that we hope is similar to the "default"). One option might be to use an enhanced 'make-pipe-process' object (as described above) as the common glue between commands in an Eshell pipeline. If you had the ability to set a process's :stdin to a "pipe-process", and then Eshell had the ability to use a "pipe-process" as an output target, I think that would get you (close to?) native pipes while still allowing one end of the pipe to be something other than a process. This doesn't help the Tramp case, since the pipe is still on your local host, but the Tramp case is hard enough that it might be best just to focus on not making the situation any worse than it already is. One final wrinkle to all this: I've been intending for quite a long time to add the ability to pipe data in Eshell *to* Lisp code; currently you can only pipe *from* Lisp code. I want to be extra careful that any changes to Eshell pipelines won't render these plans impossible. (This project has taken a long time since I want to be sure that when I eventually merge these Eshell "pseudo-processes", I get them as close to correct the first time as I can; incompatible changes down the line would be a pain.)