[bug#50239] [PATCH core-updates-frozen] gnu: diffutils: Fix signal processing.

[bug#50239] [PATCH core-updates-frozen] gnu: diffutils: Fix signal processing.

[Thiago Jung Bauermann via Guix-patches via]

diffutils has a race condition in its signal processing code which is easy
to trigger on powerpc64le-linux. More often than not, it causes the
‘colors’ test to fail and therefore the build of the package fails as well.

Add the patch proposed in Debian bug 922552 which fixes the problem.

* gnu/packages/patches/diffutils-fix-signal-processing.patch: New file.
* gnu/local.mk (dist_patch_DATA): Add it.
* gnu/packages/base.scm (diffutils)[source]: Use it.
---

Hello,

This fixes the build of diffutils on powerpc64le-linux, which currently
fails more often than not. The patch I’m adding here isn’t being
shipped by Debian and hasn’t been seen by upstream yet. I just brought
it to their attention here:

https://debbugs.gnu.org/cgi/bugreport.cgi?bug=34519#11

I’m not familiar with the diffutils code base, but FWIW I analysed
the patch and it looks very reasonable to me. To be honest I’m not
sure if it completely fixes the race condition or just makes it much
less likely to happen, but in any case I can’t hit the race condition
anymore.

In addition, since all it does is add a new call to the function which
checks and processes any pending signal, I don’t think it can cause
any harm.

Finally, this patch is based on top of the one which updates diffutils
to version 3.8:

https://issues.guix.gnu.org/50233

The fix works equally well in version 3.7 so if you think it’s not
worth updating diffutils I can rebase this patch on top of current
‘core-updates-frozen’.

 gnu/local.mk                                  |  1 +
 gnu/packages/base.scm                         |  3 +-
 .../diffutils-fix-signal-processing.patch     | 60 +++++++++++++++++++
 3 files changed, 63 insertions(+), 1 deletion(-)
 create mode 100644 gnu/packages/patches/diffutils-fix-signal-processing.patch

diff --git a/gnu/local.mk b/gnu/local.mk
index 11b002b66e72..bc385eecc592 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -962,6 +962,7 @@ dist_patch_DATA =						\
   %D%/packages/patches/desmume-gcc6-fixes.patch			\
   %D%/packages/patches/desmume-gcc7-fixes.patch			\
   %D%/packages/patches/dfu-programmer-fix-libusb.patch		\
+  %D%/packages/patches/diffutils-fix-signal-processing.patch	\
   %D%/packages/patches/diffutils-gets-undeclared.patch		\
   %D%/packages/patches/disarchive-cross-compilation.patch	\
   %D%/packages/patches/dkimproxy-add-ipv6-support.patch		\
diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
index 2c648953ae39..0e3b346b93a0 100644
--- a/gnu/packages/base.scm
+++ b/gnu/packages/base.scm
@@ -272,7 +272,8 @@ differences.")
                                 version ".tar.xz"))
             (sha256
              (base32
-              "1v4g8gi0lgakqa7iix8s4fq7lq6l92vw3rjd9wfd2rhjng8xggd6"))))
+              "1v4g8gi0lgakqa7iix8s4fq7lq6l92vw3rjd9wfd2rhjng8xggd6"))
+            (patches (search-patches "diffutils-fix-signal-processing.patch"))))
    (build-system gnu-build-system)
    (native-inputs (list perl))
    (synopsis "Comparing and merging files")
diff --git a/gnu/packages/patches/diffutils-fix-signal-processing.patch b/gnu/packages/patches/diffutils-fix-signal-processing.patch
new file mode 100644
index 000000000000..24130bd4c37a
--- /dev/null
+++ b/gnu/packages/patches/diffutils-fix-signal-processing.patch
@@ -0,0 +1,60 @@
+Author: Frédéric Bonnard <frediz@debian.org>
+
+Obtained from:
+
+https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=922552#19
+
+and slightly adapted to apply on v3.8.
+
+Fixes bug reported upstream at:
+
+https://debbugs.gnu.org/cgi/bugreport.cgi?bug=34519
+
+diff --git a/src/diff.c b/src/diff.c
+index 9938daa0c8fd..2bc443f1ca70 100644
+--- a/src/diff.c
++++ b/src/diff.c
+@@ -1453,6 +1453,8 @@ compare_files (struct comparison const *parent,
+         }
+     }
+ 
++  final_process_signals ();
++
+   /* Now the comparison has been done, if no error prevented it,
+      and STATUS is the value this function will return.  */
+ 
+diff --git a/src/diff.h b/src/diff.h
+index 27362c010fd2..28c89b0797ef 100644
+--- a/src/diff.h
++++ b/src/diff.h
+@@ -390,6 +390,7 @@ extern enum changes analyze_hunk (struct change *, lin *, lin *, lin *, lin *);
+ extern void begin_output (void);
+ extern void debug_script (struct change *);
+ extern void fatal (char const *) __attribute__((noreturn));
++extern void final_process_signals (void);
+ extern void finish_output (void);
+ extern void message (char const *, char const *, char const *);
+ extern void message5 (char const *, char const *, char const *,
+diff --git a/src/util.c b/src/util.c
+index 4348757e1507..8954197f33fc 100644
+--- a/src/util.c
++++ b/src/util.c
+@@ -237,6 +237,18 @@ process_signals (void)
+     }
+ }
+ 
++/* Process remaining signals once before exit  */
++void
++final_process_signals (void)
++{
++  static int last = 1;
++
++  if (last) {
++    process_signals ();
++    last = 0;
++  }
++}
++
+ static void
+ install_signal_handlers (void)
+ {

[bug#50239] [PATCH core-updates-frozen] gnu: diffutils: Fix signal processing.

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 5800 bytes --]

Thiago Jung Bauermann via Guix-patches via schreef op za 28-08-2021 om 13:43 [-0300]:
> [...]
> diffutils has a race condition in its signal processing code which is easy
> to trigger on powerpc64le-linux. More often than not, it causes the
> ‘colors’ test to fail and therefore the build of the package fails as well.
> [...]

Adding this patch makes sense to me too, though I don't have a powerpc64le-linux
to test this on.

> Add the patch proposed in Debian bug 922552 which fixes the problem.

Do you think upstream will have its own working patch soonish?
(See <https://debbugs.gnu.org/cgi/bugreport.cgi?bug=36488#41>, for people
who aren't Thiago Jung Bauermann.)  If so, it might make sense to wait a little
for an ‘official’ patch.

> * gnu/packages/patches/diffutils-fix-signal-processing.patch: New file.
> * gnu/local.mk (dist_patch_DATA): Add it.
> * gnu/packages/base.scm (diffutils)[source]: Use it.
> ---
> 
> Hello,
> 
> This fixes the build of diffutils on powerpc64le-linux, which currently
> fails more often than not. The patch I’m adding here isn’t being
> shipped by Debian and hasn’t been seen by upstream yet. I just brought
> it to their attention here:
> 
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=34519#11
> 
> I’m not familiar with the diffutils code base, but FWIW I analysed
> the patch and it looks very reasonable to me. To be honest I’m not
> sure if it completely fixes the race condition or just makes it much
> less likely to happen, but in any case I can’t hit the race condition
> anymore.
> 
> In addition, since all it does is add a new call to the function which
> checks and processes any pending signal, I don’t think it can cause
> any harm.
> 
> Finally, this patch is based on top of the one which updates diffutils
> to version 3.8:
> 
> https://issues.guix.gnu.org/50233
> 
> The fix works equally well in version 3.7 so if you think it’s not
> worth updating diffutils I can rebase this patch on top of current
> ‘core-updates-frozen’.
> 
>  gnu/local.mk                                  |  1 +
>  gnu/packages/base.scm                         |  3 +-
>  .../diffutils-fix-signal-processing.patch     | 60 +++++++++++++++++++
>  3 files changed, 63 insertions(+), 1 deletion(-)
>  create mode 100644 gnu/packages/patches/diffutils-fix-signal-processing.patch
> 
> diff --git a/gnu/local.mk b/gnu/local.mk
> index 11b002b66e72..bc385eecc592 100644
> --- a/gnu/local.mk
> +++ b/gnu/local.mk
> @@ -962,6 +962,7 @@ dist_patch_DATA =						\
>    %D%/packages/patches/desmume-gcc6-fixes.patch			\
>    %D%/packages/patches/desmume-gcc7-fixes.patch			\
>    %D%/packages/patches/dfu-programmer-fix-libusb.patch		\
> +  %D%/packages/patches/diffutils-fix-signal-processing.patch	\
>    %D%/packages/patches/diffutils-gets-undeclared.patch		\
>    %D%/packages/patches/disarchive-cross-compilation.patch	\
>    %D%/packages/patches/dkimproxy-add-ipv6-support.patch		\
> diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
> index 2c648953ae39..0e3b346b93a0 100644
> --- a/gnu/packages/base.scm
> +++ b/gnu/packages/base.scm
> @@ -272,7 +272,8 @@ differences.")
>                                  version ".tar.xz"))
>              (sha256
>               (base32
> -              "1v4g8gi0lgakqa7iix8s4fq7lq6l92vw3rjd9wfd2rhjng8xggd6"))))
> +              "1v4g8gi0lgakqa7iix8s4fq7lq6l92vw3rjd9wfd2rhjng8xggd6"))
> +            (patches (search-patches "diffutils-fix-signal-processing.patch"))))
>     (build-system gnu-build-system)
>     (native-inputs (list perl))
>     (synopsis "Comparing and merging files")
> diff --git a/gnu/packages/patches/diffutils-fix-signal-processing.patch b/gnu/packages/patches/diffutils-fix-signal-processing.patch
> new file mode 100644
> index 000000000000..24130bd4c37a
> --- /dev/null
> +++ b/gnu/packages/patches/diffutils-fix-signal-processing.patch
> @@ -0,0 +1,60 @@
> +Author: Frédéric Bonnard <frediz@debian.org>
> +
> +Obtained from:
> +
> +https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=922552#19
> +
> +and slightly adapted to apply on v3.8.
> +
> +Fixes bug reported upstream at:
> +
> +https://debbugs.gnu.org/cgi/bugreport.cgi?bug=34519
> +
> +diff --git a/src/diff.c b/src/diff.c
> +index 9938daa0c8fd..2bc443f1ca70 100644
> +--- a/src/diff.c
> ++++ b/src/diff.c
> +@@ -1453,6 +1453,8 @@ compare_files (struct comparison const *parent,
> +         }
> +     }
> + 
> ++  final_process_signals ();
> ++
> +   /* Now the comparison has been done, if no error prevented it,
> +      and STATUS is the value this function will return.  */
> + 
> +diff --git a/src/diff.h b/src/diff.h
> +index 27362c010fd2..28c89b0797ef 100644
> +--- a/src/diff.h
> ++++ b/src/diff.h
> +@@ -390,6 +390,7 @@ extern enum changes analyze_hunk (struct change *, lin *, lin *, lin *, lin *);
> + extern void begin_output (void);
> + extern void debug_script (struct change *);
> + extern void fatal (char const *) __attribute__((noreturn));
> ++extern void final_process_signals (void);
> + extern void finish_output (void);
> + extern void message (char const *, char const *, char const *);
> + extern void message5 (char const *, char const *, char const *,
> +diff --git a/src/util.c b/src/util.c
> +index 4348757e1507..8954197f33fc 100644
> +--- a/src/util.c
> ++++ b/src/util.c
> +@@ -237,6 +237,18 @@ process_signals (void)
> +     }
> + }
> + 
> ++/* Process remaining signals once before exit  */
> ++void
> ++final_process_signals (void)
> ++{
> ++  static int last = 1;
> ++
> ++  if (last) {
> ++    process_signals ();
> ++    last = 0;
> ++  }
> ++}
> ++
> + static void
> + install_signal_handlers (void)
> + {
> 
> 
> 

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

[bug#50239] [PATCH core-updates-frozen] gnu: diffutils: Fix signal processing.

[Thiago Jung Bauermann via Guix-patches via]

Hello Maxime,

Thank you for your review!

Em segunda-feira, 30 de agosto de 2021, às 08:43:42 -03, Maxime Devos 
escreveu:
> Thiago Jung Bauermann via Guix-patches via schreef op za 28-08-2021 om 
13:43 [-0300]:
> > [...]
> > diffutils has a race condition in its signal processing code which is
> > easy to trigger on powerpc64le-linux. More often than not, it causes
> > the ‘colors’ test to fail and therefore the build of the package fails
> > as well. [...]
> 
> Adding this patch makes sense to me too, though I don't have a
> powerpc64le-linux to test this on.

Nice, thanks!

> > Add the patch proposed in Debian bug 922552 which fixes the problem.
> 
> Do you think upstream will have its own working patch soonish?

I think so. Paul Eggert was very quick in his response and is already 
working on a fix.

> (See <https://debbugs.gnu.org/cgi/bugreport.cgi?bug=36488#41>, for people
> who aren't Thiago Jung Bauermann.)

Yeah, the discussion moved unexpectedly to a different bug report. Thanks 
for tracking it down and posting the new link.

> If so, it might make sense to wait a little for an ‘official’ patch.

Yes, I agree.

-- 
Thanks,
Thiago

[bug#50239] [PATCH core-updates-frozen] gnu: diffutils: Fix signal processing.

[Ludovic Courtès]

Hi,

Thiago Jung Bauermann <bauermann@kolabnow.com> skribis:

> This fixes the build of diffutils on powerpc64le-linux, which currently
> fails more often than not. The patch I’m adding here isn’t being
> shipped by Debian and hasn’t been seen by upstream yet. I just brought
> it to their attention here:
>
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=34519#11

Thanks for working on this!

> I’m not familiar with the diffutils code base, but FWIW I analysed
> the patch and it looks very reasonable to me. To be honest I’m not
> sure if it completely fixes the race condition or just makes it much
> less likely to happen, but in any case I can’t hit the race condition
> anymore.
>
> In addition, since all it does is add a new call to the function which
> checks and processes any pending signal, I don’t think it can cause
> any harm.
>
> Finally, this patch is based on top of the one which updates diffutils
> to version 3.8:
>
> https://issues.guix.gnu.org/50233
>
> The fix works equally well in version 3.7 so if you think it’s not
> worth updating diffutils I can rebase this patch on top of current
> ‘core-updates-frozen’.

Normally we won’t update diffutils on ‘core-updates-frozen’.

Thus, could you adjust this patch so that (1) it applies on
‘core-updates-frozen’, and (2) it doesn’t change derivations on other
platforms (thus, the patch needs to be applied from a build phase)?
Bonus point if there’s an upstream patch to use, as Maxime suggested.

TIA,
Ludo’.

[bug#50239] [PATCH core-updates-frozen] gnu: diffutils: Fix signal processing.

[Thiago Jung Bauermann via Guix-patches via]

Hi Ludo,

Em segunda-feira, 6 de setembro de 2021, às 08:55:52 -03, Ludovic Courtès 
escreveu:
> Hi,
> 
> Thiago Jung Bauermann <bauermann@kolabnow.com> skribis:
> > This fixes the build of diffutils on powerpc64le-linux, which currently
> > fails more often than not. The patch I’m adding here isn’t being
> > shipped by Debian and hasn’t been seen by upstream yet. I just brought
> > it to their attention here:
> > 
> > https://debbugs.gnu.org/cgi/bugreport.cgi?bug=34519#11
> 
> Thanks for working on this!

You’re welcome. Thank you for reviewing the patches!

> > I’m not familiar with the diffutils code base, but FWIW I analysed
> > the patch and it looks very reasonable to me. To be honest I’m not
> > sure if it completely fixes the race condition or just makes it much
> > less likely to happen, but in any case I can’t hit the race condition
> > anymore.
> > 
> > In addition, since all it does is add a new call to the function which
> > checks and processes any pending signal, I don’t think it can cause
> > any harm.
> > 
> > Finally, this patch is based on top of the one which updates diffutils
> > to version 3.8:
> > 
> > https://issues.guix.gnu.org/50233
> > 
> > The fix works equally well in version 3.7 so if you think it’s not
> > worth updating diffutils I can rebase this patch on top of current
> > ‘core-updates-frozen’.
> 
> Normally we won’t update diffutils on ‘core-updates-frozen’.

Yes, it makes sense.

> Thus, could you adjust this patch so that (1) it applies on
> ‘core-updates-frozen’, and (2) it doesn’t change derivations on other
> platforms (thus, the patch needs to be applied from a build phase)?

Sure, no problem. I’ll do that. Thanks for the tip about having to apply 
the patch from a build phase.

> Bonus point if there’s an upstream patch to use, as Maxime suggested.

There isn’t one for now. We have three alternatives if the situation 
doesn’t change by the time ‘core-updates-frozen’ gets merged:

1. Apply this patch.

2. Disable the ‘colors’ test. This is what Debian is doing. It’s a bit 
unfortunate in that we’re disabling a test that just did its job of finding 
a bug, but on the other hand the bug is minor (the terminal is left in a 
wrong state if ‘diff’ is interrupted in the middle of printing colored 
output) and easy to work around (just run ‘reset’ if that happens).

3. Do nothing. If the diffutils build fails on the CI, restart it until it 
succeeds. :-)

That is my personal order of preference, but option 1 carries a slight risk 
in that we would be shipping a patch that isn’t “battle tested”, so the 
other options are very reasonable too.

-- 
Thanks,
Thiago

[bug#50239] [PATCH core-updates-frozen v2] gnu: diffutils: Fix signal processing.

[Thiago Jung Bauermann via Guix-patches via]

diffutils has a race condition in its signal processing code which is easy
to trigger on powerpc64le-linux. More often than not, it causes the
‘colors’ test to fail and therefore the build of the package fails as well.

Add the patch proposed in Debian bug 922552 which fixes the problem.

* gnu/packages/patches/diffutils-fix-signal-processing.patch: New file.
* gnu/local.mk (dist_patch_DATA): Add it.
* gnu/packages/base.scm (diffutils)[source]: Use it.
---

Hello,

Here is the same patch rebased on top of ‘core-updates-frozen’. I didn’t
yet do it in a way that preserves derivations on other platforms because
there will be a batch of world-rebuilding changes¹, so depending on how
soon the branch is going to be merged into master, it could be a good
opportunity to land this fix now.

So far, there hasn’t been news from upstream on a fix for the problem,
but it’s been only a bit over a week since they last proposed a patch
so it’s too early to be worried about anything.

I am perfectly fine with skipping this world-rebuilding window and sending
a v3 which preserves derivations on other platforms as you suggested. Or
even not fixing this bug on ’core-updates-frozen’ if the upstream fix
doesn’t arrive by then. As I mentioned earlier, we can just hit the “try
again” button until the CI produces a successful build. :-)

I’m just sending this version to have the option on the table.

Thanks!

¹ https://lists.nongnu.org/archive/html/guix-devel/2021-09/msg00097.html

 gnu/local.mk                                  |  1 +
 gnu/packages/base.scm                         |  3 +-
 .../diffutils-fix-signal-processing.patch     | 58 +++++++++++++++++++
 3 files changed, 61 insertions(+), 1 deletion(-)
 create mode 100644 gnu/packages/patches/diffutils-fix-signal-processing.patch

diff --git a/gnu/local.mk b/gnu/local.mk
index 07e6787642e9..d37b1b577a04 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -967,6 +967,7 @@ dist_patch_DATA =						\
   %D%/packages/patches/desmume-gcc6-fixes.patch			\
   %D%/packages/patches/desmume-gcc7-fixes.patch			\
   %D%/packages/patches/dfu-programmer-fix-libusb.patch		\
+  %D%/packages/patches/diffutils-fix-signal-processing.patch	\
   %D%/packages/patches/diffutils-gets-undeclared.patch		\
   %D%/packages/patches/disarchive-cross-compilation.patch	\
   %D%/packages/patches/dkimproxy-add-ipv6-support.patch		\
diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
index 70eb135a1e41..bfe769fbc4ed 100644
--- a/gnu/packages/base.scm
+++ b/gnu/packages/base.scm
@@ -274,7 +274,8 @@ differences.")
             (sha256
              (base32
               "09isrg0isjinv8c535nxsi1s86wfdfzml80dbw41dj9x3hiad9xk"))
-            (patches (search-patches "coreutils-gnulib-tests.patch"))))
+            (patches (search-patches "coreutils-gnulib-tests.patch"
+                                     "diffutils-fix-signal-processing.patch"))))
    (build-system gnu-build-system)
    (native-inputs (list perl))
    (synopsis "Comparing and merging files")
diff --git a/gnu/packages/patches/diffutils-fix-signal-processing.patch b/gnu/packages/patches/diffutils-fix-signal-processing.patch
new file mode 100644
index 000000000000..134dd3f718bd
--- /dev/null
+++ b/gnu/packages/patches/diffutils-fix-signal-processing.patch
@@ -0,0 +1,58 @@
+Author: Frédéric Bonnard <frediz@debian.org>
+
+Obtained from:
+
+https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=922552#19
+
+Fixes bug reported upstream at:
+
+https://debbugs.gnu.org/cgi/bugreport.cgi?bug=34519
+
+diff --git a/src/diff.c b/src/diff.c
+index e2eb32437353..b574e8282dc9 100644
+--- a/src/diff.c
++++ b/src/diff.c
+@@ -1451,6 +1451,8 @@ compare_files (struct comparison const *parent,
+ 	}
+     }
+ 
++  final_process_signals ();
++
+   /* Now the comparison has been done, if no error prevented it,
+      and STATUS is the value this function will return.  */
+ 
+diff --git a/src/diff.h b/src/diff.h
+index 03daaa4a0530..e177fe600a25 100644
+--- a/src/diff.h
++++ b/src/diff.h
+@@ -390,6 +390,7 @@ extern enum changes analyze_hunk (struct change *, lin *, lin *, lin *, lin *);
+ extern void begin_output (void);
+ extern void debug_script (struct change *);
+ extern void fatal (char const *) __attribute__((noreturn));
++extern void final_process_signals (void);
+ extern void finish_output (void);
+ extern void message (char const *, char const *, char const *);
+ extern void message5 (char const *, char const *, char const *,
+diff --git a/src/util.c b/src/util.c
+index 4f4d9bb285eb..56d292de2927 100644
+--- a/src/util.c
++++ b/src/util.c
+@@ -237,6 +237,18 @@ process_signals (void)
+     }
+ }
+ 
++/* Process remaining signals once before exit  */
++void
++final_process_signals (void)
++{
++  static int last = 1;
++
++  if (last) {
++    process_signals ();
++    last = 0;
++  }
++}
++
+ static void
+ install_signal_handlers (void)
+ {

bug#50239: [PATCH core-updates-frozen] gnu: diffutils: Fix signal processing.

[Maxim Cournoyer]

Thiago Jung Bauermann <bauermann@kolabnow.com> writes:

> diffutils has a race condition in its signal processing code which is easy
> to trigger on powerpc64le-linux. More often than not, it causes the
> ‘colors’ test to fail and therefore the build of the package fails as well.
>
> Add the patch proposed in Debian bug 922552 which fixes the problem.
>
> * gnu/packages/patches/diffutils-fix-signal-processing.patch: New file.
> * gnu/local.mk (dist_patch_DATA): Add it.
> * gnu/packages/base.scm (diffutils)[source]: Use it.
> ---

Applied with commit f234f5b0e120ea8cfa850440a428c3bc211b329b to
core-updates-frozen, and diffutils was updated to 3.8 with
57fd6d511c678eeee88f254c511847a56f86f85f there as well.

Closing,

Thank you!

Maxim