Building master hangs generating lisp/eshell/esh-util-tests.log

Building master hangs generating lisp/eshell/esh-util-tests.log

[Pedro Andres Aranda Gutierrez]

[-- Attachment #1: Type: text/plain, Size: 363 bytes --]

Hi

I have retried to build master and the process hangs at
GEN      lisp/eshell/esh-util-tests.log

Best, /PA
-- 
Fragen sind nicht da, um beantwortet zu werden,
Fragen sind da um gestellt zu werden
Georg Kreisler

Headaches with a Juju log:
unit-basic-16: 09:17:36 WARNING juju.worker.uniter.operation we should run
a leader-deposed hook here, but we can't yet

[-- Attachment #2: Type: text/html, Size: 713 bytes --]

Re: Building master hangs generating lisp/eshell/esh-util-tests.log

[Eli Zaretskii]

> From: Pedro Andres Aranda Gutierrez <paaguti@gmail.com>
> Date: Wed, 12 Jun 2024 17:54:38 +0200
> 
> I have retried to build master and the process hangs at
> GEN      lisp/eshell/esh-util-tests.log

I don't understand.  First, a build doesn't include running the test
suite, so what do you mean by "the build process hangs at"?

And second, running this:

  $ make -C test lisp/eshell/esh-util-tests

doesn't hang here.  This is with the current master branch.

Re: Building master hangs generating lisp/eshell/esh-util-tests.log

[Stefan Kangas]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Pedro Andres Aranda Gutierrez <paaguti@gmail.com>
>> Date: Wed, 12 Jun 2024 17:54:38 +0200
>>
>> I have retried to build master and the process hangs at
>> GEN      lisp/eshell/esh-util-tests.log
>
> I don't understand.  First, a build doesn't include running the test
> suite, so what do you mean by "the build process hangs at"?
>
> And second, running this:
>
>   $ make -C test lisp/eshell/esh-util-tests
>
> doesn't hang here.  This is with the current master branch.

FWIW, it doesn't hang here either.

Re: Building master hangs generating lisp/eshell/esh-util-tests.log

[Andrea Corallo]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Pedro Andres Aranda Gutierrez <paaguti@gmail.com>
>> Date: Wed, 12 Jun 2024 17:54:38 +0200
>> 
>> I have retried to build master and the process hangs at
>> GEN      lisp/eshell/esh-util-tests.log
>
> I don't understand.  First, a build doesn't include running the test
> suite, so what do you mean by "the build process hangs at"?
>
> And second, running this:
>
>   $ make -C test lisp/eshell/esh-util-tests
>
> doesn't hang here.  This is with the current master branch.

I see hangs during tests on my local CI as well since about 2 3 days,
but unfortunatelly they are not very reproducible.  I'll try to gain
some more data points.

  Andrea

Re: Building master hangs generating lisp/eshell/esh-util-tests.log

[Alan Mackenzie]

Hello, Andrea.

On Thu, Jun 13, 2024 at 04:36:04 -0400, Andrea Corallo wrote:
> Eli Zaretskii <eliz@gnu.org> writes:

> >> From: Pedro Andres Aranda Gutierrez <paaguti@gmail.com>
> >> Date: Wed, 12 Jun 2024 17:54:38 +0200

> >> I have retried to build master and the process hangs at
> >> GEN      lisp/eshell/esh-util-tests.log

> > I don't understand.  First, a build doesn't include running the test
> > suite, so what do you mean by "the build process hangs at"?

> > And second, running this:

> >   $ make -C test lisp/eshell/esh-util-tests

> > doesn't hang here.  This is with the current master branch.

> I see hangs during tests on my local CI as well since about 2 3 days,
> but unfortunatelly they are not very reproducible.  I'll try to gain
> some more data points.

I've just updated my master repo.  It built OK, and on running make
check, two erc test files hung.  After killing them (after ~2 minutes run
time), I got the following display on my terminal:

SUMMARY OF TEST RESULTS
-----------------------
Files examined: 518
Ran 7858 tests, 28 failed to run, 7659 results as expected, 1 unexpected,
198 skipped
2 files did not finish:
  lisp/eshell/esh-proc-tests.log
  lisp/eshell/esh-ext-tests.log
1 files contained unexpected results:
  lisp/minibuffer-tests.log

I got basically the same on running make check in one of my other repos
which I'd just updated.  So the problem appears to be reproducible for
me.

>   Andrea

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Building master hangs generating lisp/eshell/esh-util-tests.log

[Andrea Corallo]

Alan Mackenzie <acm@muc.de> writes:

> Hello, Andrea.
>
> On Thu, Jun 13, 2024 at 04:36:04 -0400, Andrea Corallo wrote:
>> Eli Zaretskii <eliz@gnu.org> writes:
>
>> >> From: Pedro Andres Aranda Gutierrez <paaguti@gmail.com>
>> >> Date: Wed, 12 Jun 2024 17:54:38 +0200
>
>> >> I have retried to build master and the process hangs at
>> >> GEN      lisp/eshell/esh-util-tests.log
>
>> > I don't understand.  First, a build doesn't include running the test
>> > suite, so what do you mean by "the build process hangs at"?
>
>> > And second, running this:
>
>> >   $ make -C test lisp/eshell/esh-util-tests
>
>> > doesn't hang here.  This is with the current master branch.
>
>> I see hangs during tests on my local CI as well since about 2 3 days,
>> but unfortunatelly they are not very reproducible.  I'll try to gain
>> some more data points.
>
> I've just updated my master repo.  It built OK, and on running make
> check, two erc test files hung.  After killing them (after ~2 minutes run
> time), I got the following display on my terminal:
>
> SUMMARY OF TEST RESULTS
> -----------------------
> Files examined: 518
> Ran 7858 tests, 28 failed to run, 7659 results as expected, 1 unexpected,
> 198 skipped
> 2 files did not finish:
>   lisp/eshell/esh-proc-tests.log
>   lisp/eshell/esh-ext-tests.log
> 1 files contained unexpected results:
>   lisp/minibuffer-tests.log
>
> I got basically the same on running make check in one of my other repos
> which I'd just updated.  So the problem appears to be reproducible for
> me.

Right I think is reproducible for me now, I see in my CI the first
commit that makes it hang is:

3199b8584e4583a4b74263f5704a21c3431658c2
Author:     Dmitry Gutov <dmitry@gutov.dev>
AuthorDate: Tue Jun 11 21:02:57 2024 +0300
Commit:     Dmitry Gutov <dmitry@gutov.dev>
CommitDate: Tue Jun 11 21:03:19 2024 +0300

Parent:     5ecff95993d Refactor tramp-*-process-file functions
Merged:     emacs1 emacs2 emacs3 emacs4
Follows:    emacs-29.1.90 (171714)

read_and_insert_process_output: Insert before markers

* src/insdel.c (adjust_markers_for_insert): Make non-static.

* src/lisp.h: Add declaration for it.

* src/process.c (read_and_insert_process_output): Use it here.
And also call insert_1_both with BEFORE_MARKERS=t, for
compatibility with internal-default-process-filter (bug#66020).

If I apply the following:

===========

modified   src/process.c
@@ -6423,9 +6423,9 @@ read_and_insert_process_output (struct Lisp_Process *p, char *buf,
       specbind (Qinhibit_modification_hooks, Qt);
       decode_coding_c_string (process_coding,
                               (unsigned char *) buf, nread, curbuf);
-      adjust_markers_for_insert (PT, PT_BYTE,
-                                 PT + process_coding->produced_char,
-                                 PT_BYTE + process_coding->produced, true);
+      /* adjust_markers_for_insert (PT, PT_BYTE, */
+      /*                                PT + process_coding->produced_char, */
+      /*                                PT_BYTE + process_coding->produced, true); */
       unbind_to (count1, Qnil);

       read_process_output_set_last_coding_system (p, process_coding);


===========

my make check does not hang.

Ccing Dmitry.

Thanks

  Andrea

Re: Building master hangs generating lisp/eshell/esh-util-tests.log

[Steven Allen]

Andrea Corallo <acorallo@gnu.org> writes:
> If I apply the following:
>
> ===========
>
> modified   src/process.c
> @@ -6423,9 +6423,9 @@ read_and_insert_process_output (struct Lisp_Process *p, char *buf,
>        specbind (Qinhibit_modification_hooks, Qt);
>        decode_coding_c_string (process_coding,
>                                (unsigned char *) buf, nread, curbuf);
> -      adjust_markers_for_insert (PT, PT_BYTE,
> -                                 PT + process_coding->produced_char,
> -                                 PT_BYTE + process_coding->produced, true);
> +      /* adjust_markers_for_insert (PT, PT_BYTE, */
> +      /*                                PT + process_coding->produced_char, */
> +      /*                                PT_BYTE + process_coding->produced, true); */
>        unbind_to (count1, Qnil);
>
>        read_process_output_set_last_coding_system (p, process_coding);
>
>
> ===========
>
> my make check does not hang.

I've tested this and it seems to fix bug#71525 as well (looks like the
same bug).

Re: Building master hangs generating lisp/eshell/esh-util-tests.log

[Dmitry Gutov]

Hi Andrea,

On 13/06/2024 16:14, Andrea Corallo wrote:
> Right I think is reproducible for me now, I see in my CI the first
> commit that makes it hang is:

Do you use macOS? I haven't been able to reproduce it on Linux so far.

Or could this be specific to nativecomp? Which would be weird, but...

Re: Building master hangs generating lisp/eshell/esh-util-tests.log

[Andrea Corallo]

Dmitry Gutov <dmitry@gutov.dev> writes:

> Hi Andrea,
>
> On 13/06/2024 16:14, Andrea Corallo wrote:
>> Right I think is reproducible for me now, I see in my CI the first
>> commit that makes it hang is:
>
> Do you use macOS? I haven't been able to reproduce it on Linux so far.
>
> Or could this be specific to nativecomp? Which would be weird, but...

Hi Dmitry,

this is on GNU/Linux configured with:

--with-native-compilation=yes --with-gnutls=ifavailable

I tried now with --with-native-compilation=no but still see the issue.

  Andrea

Re: Building master hangs generating lisp/eshell/esh-util-tests.log

[Dmitry Gutov]

Hi Andrea,

On 14/06/2024 01:55, Andrea Corallo wrote:
> this is on GNU/Linux configured with:
> 
> --with-native-compilation=yes --with-gnutls=ifavailable
> 
> I tried now with --with-native-compilation=no but still see the issue.

Thanks, I got it now.

See the patch I just posted to 
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=71525#41

Re: Building master hangs generating lisp/eshell/esh-util-tests.log

[Pedro Andres Aranda Gutierrez]

[-- Attachment #1: Type: text/plain, Size: 708 bytes --]

This is on a Ubuntu 22.04 server VM where I build emacs

Best, /PA

On Thu, 13 Jun 2024 at 22:30, Dmitry Gutov <dmitry@gutov.dev> wrote:

> Hi Andrea,
>
> On 13/06/2024 16:14, Andrea Corallo wrote:
> > Right I think is reproducible for me now, I see in my CI the first
> > commit that makes it hang is:
>
> Do you use macOS? I haven't been able to reproduce it on Linux so far.
>
> Or could this be specific to nativecomp? Which would be weird, but...
>


-- 
Fragen sind nicht da, um beantwortet zu werden,
Fragen sind da um gestellt zu werden
Georg Kreisler

Headaches with a Juju log:
unit-basic-16: 09:17:36 WARNING juju.worker.uniter.operation we should run
a leader-deposed hook here, but we can't yet

[-- Attachment #2: Type: text/html, Size: 1293 bytes --]

Re: Building master hangs generating lisp/eshell/esh-util-tests.log

[Jim Porter]

On 6/12/2024 8:54 AM, Pedro Andres Aranda Gutierrez wrote:
> I have retried to build master and the process hangs at
> GEN      lisp/eshell/esh-util-tests.log

This works for me locally. Could you try `make boostrap` and then re-run 
the tests? Sometimes the tests break on incremental builds if a macro 
definition changed.

Re: Building master hangs generating lisp/eshell/esh-util-tests.log

[Pedro Andres Aranda Gutierrez]

[-- Attachment #1: Type: text/plain, Size: 995 bytes --]

I will retry. Just to make it clear. I have adapted the Debian build
infrastructure to compile emacs master on a VM and then transfer the
resulting .deb to my Ubuntu. As a part of the build process, tests are run.
Failing the tests will not abort the package generation process, my only
trouble is when a test hangs..
Best, /PA


On Wed, 12 Jun 2024 at 21:09, Jim Porter <jporterbugs@gmail.com> wrote:

> On 6/12/2024 8:54 AM, Pedro Andres Aranda Gutierrez wrote:
> > I have retried to build master and the process hangs at
> > GEN      lisp/eshell/esh-util-tests.log
>
> This works for me locally. Could you try `make boostrap` and then re-run
> the tests? Sometimes the tests break on incremental builds if a macro
> definition changed.
>


-- 
Fragen sind nicht da, um beantwortet zu werden,
Fragen sind da um gestellt zu werden
Georg Kreisler

Headaches with a Juju log:
unit-basic-16: 09:17:36 WARNING juju.worker.uniter.operation we should run
a leader-deposed hook here, but we can't yet

[-- Attachment #2: Type: text/html, Size: 1615 bytes --]

Re: Building master hangs generating lisp/eshell/esh-util-tests.log

[Jim Porter]

On 6/12/2024 11:14 PM, Pedro Andres Aranda Gutierrez wrote:
> I will retry. Just to make it clear. I have adapted the Debian build 
> infrastructure to compile emacs master on a VM and then transfer the 
> resulting .deb to my Ubuntu. As a part of the build process, tests are run.
> Failing the tests will not abort the package generation process, my only 
> trouble is when a test hangs..
> Best, /PA

No need for you to retry. Now that I pulled the latest Emacs from Git, I 
can reproduce this as well, and it sounds like we've already found the 
offending commit. (I'm not sure yet whether the problem is in that 
change or if it just revealed a latent Eshell bug.)

Re: Building master hangs generating lisp/eshell/esh-util-tests.log

[Dmitry Gutov]

On 13/06/2024 22:19, Jim Porter wrote:
> On 6/12/2024 11:14 PM, Pedro Andres Aranda Gutierrez wrote:
>> I will retry. Just to make it clear. I have adapted the Debian build 
>> infrastructure to compile emacs master on a VM and then transfer the 
>> resulting .deb to my Ubuntu. As a part of the build process, tests are 
>> run.
>> Failing the tests will not abort the package generation process, my 
>> only trouble is when a test hangs..
>> Best, /PA
> 
> No need for you to retry. Now that I pulled the latest Emacs from Git, I 
> can reproduce this as well, and it sounds like we've already found the 
> offending commit. (I'm not sure yet whether the problem is in that 
> change or if it just revealed a latent Eshell bug.)

The change wasn't supposed to change observable behavior, so no change 
in Eshell should be required.

I wish someone could help me reproduce it though.

   make -C test lisp/eshell/esh-util-tests

finishes just fine over here, no matter how many times I try. Is it 
perhaps OS-specific? I'm testing this on GNU/Linux now, whereas 
something like macOS might plausibly go down the 'else' clause 
read_and_insert_process_output more often...

Re: Building master hangs generating lisp/eshell/esh-util-tests.log

[Jim Porter]

On 6/13/2024 1:29 PM, Dmitry Gutov wrote:
> The change wasn't supposed to change observable behavior, so no change 
> in Eshell should be required.

Good to know, thanks.

> I wish someone could help me reproduce it though.
> 
>    make -C test lisp/eshell/esh-util-tests
> 
> finishes just fine over here, no matter how many times I try. Is it 
> perhaps OS-specific? I'm testing this on GNU/Linux now, whereas 
> something like macOS might plausibly go down the 'else' clause 
> read_and_insert_process_output more often...

The email subject is a misnomer, I believe: 'esh-util-tests' is likely 
the file *before* the hanging one. Try 'esh-proc-tests' instead. From 
poking around a bit, one of the hanging tests is 
'esh-proc-test/output/remote-redirect' in that file.

Re: Building master hangs generating lisp/eshell/esh-util-tests.log

[Dmitry Gutov]

On 13/06/2024 23:35, Jim Porter wrote:
>> I wish someone could help me reproduce it though.
>>
>>    make -C test lisp/eshell/esh-util-tests
>>
>> finishes just fine over here, no matter how many times I try. Is it 
>> perhaps OS-specific? I'm testing this on GNU/Linux now, whereas 
>> something like macOS might plausibly go down the 'else' clause 
>> read_and_insert_process_output more often...
> 
> The email subject is a misnomer, I believe: 'esh-util-tests' is likely 
> the file *before* the hanging one. Try 'esh-proc-tests' instead. From 
> poking around a bit, one of the hanging tests is 
> 'esh-proc-test/output/remote-redirect' in that file.

That makes more sense, thanks. It did kind of freeze once when I tried 
it a few times.

Interactively, esh-proc-test/output/remote-redirect just fails with

     Check that redirecting stdout for a remote process works.
     Command logs: command: "*echo hello > #<eshell-temp-buffer>"

     (ert-test-failed
      ((should (eshell-match-output regexp)) :form
       (eshell-match-output "\\`\\'") :value nil :explanation
       (mismatched-output (command "*echo hello > #<eshell-temp-buffer>\n")
                          (output "echo: external command not found\n")
                          (regexp "\\`\\'"))))

But I guess it infloops in the terminal somehow...