Jim,

Thanks for the explanation.

Well, it looks like I keep running against issues that would likely be gone
if https://debbugs.gnu.org/cgi/bugreport.cgi?bug=57635 is addressed.
Unfortunately I do not have much confidence in my ability to help ,as it
would take me a long time to even get to the point of my Elisp to start
attempting.

On the limited scope of this output issue: Should I create a bug out of
this, or do you want to go ahead and report it? (or something else)

Thanks,
Milan

PS: I needed the stdout and stderr redirection to behave properly to be
able to finish my Org-babel ob-eshell project I started a few days ago.
There is already an existing ob-eshell, but it does not work for me, so I
started a re-do.  (I mostly want :results output to work). I see now that
effort is on a backburner for now until the redirection works.





On Wed, Nov 23, 2022 at 3:30 PM Jim Porter <jporterbugs@gmail.com> wrote:

> On 11/23/2022 10:56 AM, Milan Zimmermann wrote:
> > I would like to check here before reporting an Eshell bug.
> >
> > A script named 'redirect-echo.esh' with two echo commands.
> >
> > Emacs 29: Sourcing it and redirecting output to a file, results in the
> > first echo string ('hello') showing in eshell, only the second
> > ('there')(presumably because it is last) showing in the output file:
> [snip]
> >   I believe this is a bug but I am not sure about Eshell's intended
> > features, so I wanted to check first.
>
> This is definitely a bug. I think the best way to fix it though would be
> to first replace 'eshell-do-eval', since it's a bit limited in what Lisp
> forms it can manage correctly.
>
> (Warning: the following is based on my memory of looking at this a few
> months ago, so I might have misremembered some details. If so, sorry
> about that.)
>
> Currently, Eshell manages output handles in a fairly tricky way: many
> internal Eshell forms automatically try to close the handles, so to get
> around this, other Eshell forms increment the handles' refcount so that
> closing just decrements that count instead of actually closing it.
> That's a fine strategy in general, but the increment-refcount
> ('eshell-protect') and decrement-refcount-and-close
> ('eshell-close-handles') are called very far from each other in the
> code, so the logic gets pretty hard to follow, leading to bugs like this.
>
> I think the best way to fix this would be to put the increment and
> decrement code in the same spot, so that you'd do something like so:
>
>    (unwind-protect
>        (progn
>          (eshell-protect)
>          (do-stuff))
>      (eshell-close-handles))
>
> Unfortunately, if 'do-stuff' calls a deferred command[1], Eshell ends up
> evaluating 'eshell-close-handles' immediately, instead of when
> 'do-stuff' actually finishes. That means that either a) we'd need to fix
> this without using 'unwind-protect', which would be painful to get
> right, or b) we'd need to make 'unwind-protect' work as expected. (b) is
> effectively bug#57635[2], I think. (That bug suggests using
> generator.el's internals to drive Eshell's iterative evaluation.)
>
> Bug#57635 is a pretty big change too, but I think it would greatly help
> with the overall reliability of Eshell.
>
> [1] Basically, a command that should yield execution back to the rest of
> Emacs while waiting on some background task (e.g. a subprocess).
>
> [2] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=57635
>