Reducing "You found a bug" reports

Reducing "You found a bug" reports

[Ian Eure]

There’s a steady number of bug reports generated by the "You found 
a bug" message which happens during `guix pull's.  The 
overwhelming majority of these reports are caused by networking 
problems or the Guix infrastructure being unreliable or 
overloaded.  Many of these were submitted during the recent 
guix.gnu.org downtime.

Some of these that I see:

55066
62023
62830
61520
58309

...I’m sure there are many more.

Is there some way for this code to be smarter about when it prints 
the "report a bug" message, so it doesn’t tell users to report 
bugs when none exist?  Is there a way for it to notice that the 
problem is related to networking, and tell the users to try again 
in a little while?  Is it worth removing the "report a bug" 
message entirely?

It doesn’t feel great to tell users to report a bug for things 
that aren’t bugs.  They’re either closed, or never followed up on; 
it’s a poor experience on both ends.

Thanks,

  — Ian

Re: Reducing "You found a bug" reports

[Ludovic Courtès]

Hi,

Ian Eure <ian@retrospec.tv> skribis:

> Is there some way for this code to be smarter about when it prints the
> "report a bug" message, so it doesn’t tell users to report bugs when
> none exist?  Is there a way for it to notice that the problem is
> related to networking, and tell the users to try again in a little
> while?  Is it worth removing the "report a bug" message entirely?
>
> It doesn’t feel great to tell users to report a bug for things that
> aren’t bugs.  They’re either closed, or never followed up on; it’s a
> poor experience on both ends.

I agree, it’s pretty bad.

I’m fine removing the “report a bug” message, maybe replacing it with
some clearer diagnostic and suggestion?  WDYT?

Thanks,
Ludo’.

Re: Reducing "You found a bug" reports

[Felix Lechner via Development of GNU Guix and the GNU System distribution.]

Hi Ludo' & Ian,

On Mon, Jun 17 2024, Ludovic Courtès wrote:

> I’m fine removing the “report a bug” message [...] WDYT?

Just a quick side note that some members in our community (not I) are
offended by the word "bug" to describe software defects.  Perhaps here
is a chance to replace it?

Kind regards
Felix, a beekeeper

Re: Reducing "You found a bug" reports

[Ricardo Wurmus]

Felix Lechner via "Development of GNU Guix and the GNU System distribution." <guix-devel@gnu.org> writes:

> On Mon, Jun 17 2024, Ludovic Courtès wrote:
>
>> I’m fine removing the “report a bug” message [...] WDYT?
>
> Just a quick side note that some members in our community (not I) are
> offended by the word "bug" to describe software defects.  Perhaps here
> is a chance to replace it?

I'm a big fan of bugs (the crawling kind, and based on my track record
apparently also the other kind).  The traditional use is about *dead*
bugs gumming up the works.  There is no insecticidal connotation;
removing bugs from open (...free?) machines is a necessity because they
have an unfortunate propensity for getting themselves into crevices that
they can't get out of.  Poor things.

If I was to drop the term "bug" I'd only do it to spite Thomas Edison,
whose letter appears to be the first documented case of using the term
in association with engineering defects.

-- 
Ricardo, a gardener

Re: Reducing "You found a bug" reports

[Simon Tournier]

Hi,

On Mon, 17 Jun 2024 at 14:59, Ludovic Courtès <ludo@gnu.org> wrote:

>> It doesn’t feel great to tell users to report a bug for things that
>> aren’t bugs.  They’re either closed, or never followed up on; it’s a
>> poor experience on both ends.
>
> I agree, it’s pretty bad.
>
> I’m fine removing the “report a bug” message, maybe replacing it with
> some clearer diagnostic and suggestion?  WDYT?

Could we detect if it comes from networking?  Somehow catch the error
and run some code that checks stuff and so report more “accurate”
messages?

Well, easier to say… :-)

Cheers,
simon

Re: Reducing "You found a bug" reports

[Ludovic Courtès]

Hi,

Simon Tournier <zimon.toutoune@gmail.com> skribis:

> On Mon, 17 Jun 2024 at 14:59, Ludovic Courtès <ludo@gnu.org> wrote:
>
>>> It doesn’t feel great to tell users to report a bug for things that
>>> aren’t bugs.  They’re either closed, or never followed up on; it’s a
>>> poor experience on both ends.
>>
>> I agree, it’s pretty bad.
>>
>> I’m fine removing the “report a bug” message, maybe replacing it with
>> some clearer diagnostic and suggestion?  WDYT?
>
> Could we detect if it comes from networking?  Somehow catch the error
> and run some code that checks stuff and so report more “accurate”
> messages?

Someone™ would need to analyze some of the reports we have (some of them
have already been commented on) to determine what happened and whether
that is something we could diagnose differently.

Ludo’.

Re: Reducing "You found a bug" reports

[Simon Tournier]

Hi,

On Sun, 21 Jul 2024 at 15:16, Ludovic Courtès <ludo@gnu.org> wrote:

>>> I’m fine removing the “report a bug” message, maybe replacing it with
>>> some clearer diagnostic and suggestion?  WDYT?
>>
>> Could we detect if it comes from networking?  Somehow catch the error
>> and run some code that checks stuff and so report more “accurate”
>> messages?
>
> Someone™ would need to analyze some of the reports we have (some of them
> have already been commented on) to determine what happened and whether
> that is something we could diagnose differently.

Over the years, I have spent some time in answering to some of these bug
reports; see [1] for one instance of an attempt to be this Someone™ ;-).

More than often, the issue is transient, hard if not impossible to
reproduce and thus hard to analyze.  Even, when it happens for me,
sometimes I say ok let try to collect some information for helping in
debugging that but then just running again “guix pull” makes it pass.

Excluding the bug between the keyboard and the chair, most of the time
it seems coming from either user’s network, or either substitute
servers.  Most of the time the bang is transient.

Bah I do not know exactly what or how – otherwise I would have proposed
a patch or something ;-) – but I think we need to “instrument“ the
command “guix pull” in order to collect a trace for then diagnosing.

Somehow, maybe it could be a good application of Maxim’s proposal for
logging [2].

Well, instead of an “useless” backtrace, it would be more helpful to
have a kind of log or coredump; maybe something under /var/log/guix/.

I think the next step is not « Someone™ would need to analyze some of
the reports » because the reports often provide only the same “useless“
backtrace, so instead the next step seems: « Someone™ would need to
implement a better way for generating good reports ». ;-)

Cheers,
simon

1: collection of “guix pull“ bug reports
Simon Tournier <zimon.toutoune@gmail.com>
Wed, 23 Aug 2023 18:17:20 +0200
id:86jztl20of.fsf@gmail.com
https://lists.gnu.org/archive/html/guix-devel/2023-08
https://yhetil.org/guix/86jztl20of.fsf@gmail.com

2: [bug#68946] [RFC PATCH 0/1] Add logging capability to Guix
Maxim Cournoyer <maxim.cournoyer@gmail.com>
Mon, 05 Feb 2024 23:12:00 -0500
id:cover.1707192720.git.maxim.cournoyer@gmail.com
https://issues.guix.gnu.org/68946
https://issues.guix.gnu.org/msgid/cover.1707192720.git.maxim.cournoyer@gmail.com
https://yhetil.org/guix/cover.1707192720.git.maxim.cournoyer@gmail.com

Re: Reducing "You found a bug" reports

[Suhail Singh]

Simon Tournier <zimon.toutoune@gmail.com> writes:

> More than often, the issue is transient, hard if not impossible to
> reproduce and thus hard to analyze.  Even, when it happens for me,
> sometimes I say ok let try to collect some information for helping in
> debugging that but then just running again “guix pull” makes it pass.

Doesn't that suggest, that it may help to reword the below:

#+caption: <https://issues.guix.gnu.org/55066#0-lineno145>
#+begin_quote
  guix pull: error: You found a bug: the program '/gnu/store/mpfp9nrzifhp3r5s3bv05b8xal5aa44f-compute-guix-derivation'
  failed to compute the derivation for Guix (version: "e32cc011bbe899fda432906776702f74fa6b1450"; system: "x86_64-linux";
  host version: "eb34ff16cc9038880e87e1a58a93331fca37ad92"; pull-version: 1).
  Please report the COMPLETE output above by email to <bug-guix@gnu.org>.
#+end_quote

So that we encourage users to try a few times (perhaps possibly with
--fallback), and in case of repeat failures to then submit a bug report
to <bug-guix@gnu.org>.  What about something like:

#+begin_example
  guix pull: error: An error occurred: the program '/gnu/store/mpfp9nrzifhp3r5s3bv05b8xal5aa44f-compute-guix-derivation'
  failed to compute the derivation for Guix (version: "e32cc011bbe899fda432906776702f74fa6b1450"; system: "x86_64-linux";
  host version: "eb34ff16cc9038880e87e1a58a93331fca37ad92"; pull-version: 1).

  Please try running "guix pull" again.  In case of repeated failure,
  please try passing "--fallback" to "guix pull".  In case that still
  doesn't resolve the error, please report the COMPLETE output above by
  email to <bug-guix@gnu.org>.
#+end_example

I.e., perhaps the first step is simply to reword above so that we don't
recommend that users submit a bug report on their first (possibly
transient) failure.

-- 
Suhail

Re: Reducing "You found a bug" reports

[Attila Lendvai]

> Just a quick side note that some members in our community (not I) are
> offended by the word "bug" to describe software defects.


welcome to the internet of billions of people, where the cost of expressing one's offence doesn't cost anything anymore, and there are dozens of offended people even for the most innocent of statements.

and with the advent of LLMs, it can soon become a kind of a security hole for intellectual DoS attacks -- if not already.

not sure how to keep the SNR high, but the communities will need to adapt.

--
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“Knowledge will forever govern ignorance: And a people who mean to be their own Governors, must arm themselves with the power which knowledge gives.”
	— James Madison (1751–1836), 'Epilogue: Securing the Republic' (1822)

Re: Reducing "You found a bug" reports

[chris]

Politely, my relatively-new-user observations about `guix pull` and home/system reconfigure commands,
 1. The commands, in my experience, always fail with network troubles, and the commands must be re-issued multiple times for
things to complete. Other internet-requesting utilities such as wget, curl, yt-dlp, pacman, apk and mpv do not have these
issues,
 2. The commands, based on shell output, appear to download the same versions of the same packages multiple times,
 3. The commands, based on shell output, appear to request the same substitute services over and again multiple times in a row
and they appear to do so at multiple different times within the same process. Perhaps the output messages lack details to inform the user, but to the user these appear as un-necessary repetition,
 4. The commands usually print correct localised output in the shell process, but less frequently, question marks are printed
instead of text, such as like this '????'

For a new user, the guix experience would be more convincing if the above issues were not present.