From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Jim Porter Newsgroups: gmane.emacs.devel Subject: Re: Checking if this is a Eshell bug in emacs 29: Should eshell redirect all output in esh script to stdout Date: Wed, 23 Nov 2022 12:30:45 -0800 Message-ID: <82bcaf3d-29a6-c7be-46e9-edafc9c33de1@gmail.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="25328"; mail-complaints-to="usenet@ciao.gmane.io" To: Milan Zimmermann , emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Wed Nov 23 21:32:02 2022 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1oxwPY-0006KQ-W1 for ged-emacs-devel@m.gmane-mx.org; Wed, 23 Nov 2022 21:32:01 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oxwOd-0002gI-DU; Wed, 23 Nov 2022 15:31:03 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oxwOb-0002g0-C3 for emacs-devel@gnu.org; Wed, 23 Nov 2022 15:31:01 -0500 Original-Received: from mail-pj1-x1033.google.com ([2607:f8b0:4864:20::1033]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1oxwOO-0004OL-Sx for emacs-devel@gnu.org; Wed, 23 Nov 2022 15:31:01 -0500 Original-Received: by mail-pj1-x1033.google.com with SMTP id y14-20020a17090a2b4e00b002189a1b84d4so2848007pjc.2 for ; Wed, 23 Nov 2022 12:30:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:mime-version:date:message-id:from:to:cc :subject:date:message-id:reply-to; bh=xQXlurJGZueF1tKH4oNm0UBntvW+K0nutU1ix6YHTAk=; b=GjdWKcydySb3/oMSrrChhpDI3Gxj5LOKW2QMlYHNmBoXr58lTYNvXXcFTgiKKmS+C+ EAbzzVUnlCk1Nj24tMDIOVaU3idYlQf17xQsHpx4nUCWqy5g7zP0R79nhgfBujLBL07Y oziPc47L+bf42lZaMlP1BaLtEv83o4oEOyuY84SLQiLu2GaRQjYIgokM7auAUjTFYn7f ybKC2Xo1+7GNcaNAwGiXfAdPw6EiSIsfTClqlH2ybKQUGVcFctmHEXIEKqMpQ8Eie6rv U51GAhZ1Ufr8haUckuOpKQJA9GgjAXQadAZACXFf4lpMM04aW4yQq3UULpKzXZJ5grXo JPFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=xQXlurJGZueF1tKH4oNm0UBntvW+K0nutU1ix6YHTAk=; b=58dIAk+nH7+hL+YDcUB7QkWUNuf38+qy0l0lkX26fiZgyjLakGRHsjoUePiGKqypqo WulBgwyAnKBwqGCStx3jzE/mMEeyHyV/wTKrdrGlM7FtJ1WbnOKOa0SMIshKzY0GUNMd KuzwJdostT23T7T37ADasZ75JHgyXhor0h6I2mYrNDcYbKQMY1aauoYPno9Zvj/djpPY sLAZIlm2S+HNh+CGaYZIfGLVG5bPRS+6fqT0GXMdRqfmLDJOnxkA2qmTN1pCsx6URehJ OhNbMahhHbnNsdbyWH8hEycZefSR66aZvjnOwwKdRif4Dk94tr/uNZSvwUJKt94bRS3+ RLYQ== X-Gm-Message-State: ANoB5pnPOYqGZ7YJaikmKOlvpxdIFlmtlRvl9ij8N0V6QRDSjgs9Oo8g C1VnPf3fXv65/N47VlPc1Pg= X-Google-Smtp-Source: AA0mqf4PdEJgJp7dz1PbLbkfjPIl1h0i8ZG+X8tHwXaSMbdOAnbzJUKVs9x0/cpnlEaGYcyVF81B+A== X-Received: by 2002:a17:903:41c8:b0:188:f0d7:ffbe with SMTP id u8-20020a17090341c800b00188f0d7ffbemr10407344ple.27.1669235447193; Wed, 23 Nov 2022 12:30:47 -0800 (PST) Original-Received: from [192.168.1.2] (cpe-76-168-148-233.socal.res.rr.com. [76.168.148.233]) by smtp.googlemail.com with ESMTPSA id p1-20020a170902e74100b00178b6ccc8a0sm2488415plf.51.2022.11.23.12.30.46 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 23 Nov 2022 12:30:46 -0800 (PST) Content-Language: en-US In-Reply-To: Received-SPF: pass client-ip=2607:f8b0:4864:20::1033; envelope-from=jporterbugs@gmail.com; helo=mail-pj1-x1033.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:300408 Archived-At: On 11/23/2022 10:56 AM, Milan Zimmermann wrote: > I would like to check here before reporting an Eshell bug. > > A script named 'redirect-echo.esh' with two echo commands. > > Emacs 29: Sourcing it and redirecting output to a file, results in the > first echo string ('hello') showing in eshell, only the second > ('there')(presumably because it is last) showing in the output file: [snip] >  I believe this is a bug but I am not sure about Eshell's intended > features, so I wanted to check first. This is definitely a bug. I think the best way to fix it though would be to first replace 'eshell-do-eval', since it's a bit limited in what Lisp forms it can manage correctly. (Warning: the following is based on my memory of looking at this a few months ago, so I might have misremembered some details. If so, sorry about that.) Currently, Eshell manages output handles in a fairly tricky way: many internal Eshell forms automatically try to close the handles, so to get around this, other Eshell forms increment the handles' refcount so that closing just decrements that count instead of actually closing it. That's a fine strategy in general, but the increment-refcount ('eshell-protect') and decrement-refcount-and-close ('eshell-close-handles') are called very far from each other in the code, so the logic gets pretty hard to follow, leading to bugs like this. I think the best way to fix this would be to put the increment and decrement code in the same spot, so that you'd do something like so: (unwind-protect (progn (eshell-protect) (do-stuff)) (eshell-close-handles)) Unfortunately, if 'do-stuff' calls a deferred command[1], Eshell ends up evaluating 'eshell-close-handles' immediately, instead of when 'do-stuff' actually finishes. That means that either a) we'd need to fix this without using 'unwind-protect', which would be painful to get right, or b) we'd need to make 'unwind-protect' work as expected. (b) is effectively bug#57635[2], I think. (That bug suggests using generator.el's internals to drive Eshell's iterative evaluation.) Bug#57635 is a pretty big change too, but I think it would greatly help with the overall reliability of Eshell. [1] Basically, a command that should yield execution back to the rest of Emacs while waiting on some background task (e.g. a subprocess). [2] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=57635