Reducing default verbosity

Reducing default verbosity

[Ricardo Wurmus]

Hi Guix,

for the release after 0.14.0 I’d like us to implement changes to the
default verbosity of Guix commands.

Here are a few things that I think would make for better defaults:

* Reduce package downloads from three lines to one line.  The
  “Downloading URL” line is only important when things go wrong.
* Reduce precision in percentages.  The extra digit and the period
  contribute to visual noise.
* Widen progress bar, so that it changes more quickly.

  Before:
  
    Downloading https://berlin.guixsd.org/nar/gzip/r43434523452345…-zlib-1.2.11…
     zlib-1.2.11    151KiB      12.6MiB/s  00:00 [###############] 100.0%

    Downloading https://berlin.guixsd.org/nar/gzip/rlol…-libpng-1.6.29…
     libpng-1.6.29  377KiB      10.6MiB/s  00:00 [#############  ]  90.0%

  After:

    zlib-1.2.11     151KiB  00:00 [##############################] 100%  12.6MiB/s
    libpng-1.6.29   377KiB  00:00 [#########################     ]  90%  10.6MiB/s

* Redirect all build output to log files; as we can’t generally estimate
  progress I’d use a spinner and maybe display the name of the current
  build phase and the number of build phases that are left.

  Building openldap-2.4.45 (2/6)  [‐----->-----------------------]
  Building openldap-2.4.45 (2/6)  [‐-------------->--------------]
  Building openldap-2.4.45 (3/6)  [‐----------------------->-----]
  Building openldap-2.4.45 (4/6)  [‐-----------------<-----------]
  Building openldap-2.4.45 (4/6)  [-------------<----------------]

  There are some pretty single character unicode spinners here:
    https://github.com/pmonks/spinner/blob/master/src/spinner/core.clj

* Remove double messages that are printed by the daemon and the Guix
  client, e.g. hash mismatches.  Here’s an example of this:

--8<---------------cut here---------------start------------->8---
rekado in ~: guix build guile-sjson
substitute: updating list of substitutes from 'https://mirror.hydra.gnu.org'... 100.0%
substitute: updating list of substitutes from 'https://mirror.hydra.gnu.org'... 100.0%
@ build-started /gnu/store/f4a56cwcb45mkd8vgpjprx06rs4vqhws-sjson-0.2.1.tar.gz.drv - x86_64-linux /var/log/guix/drvs/f4//a56cwcb45mkd8vgpjprx06rs4vqhws-sjson-0.2.1.tar.gz.drv.bz2

Starting download of /gnu/store/59k6bgh00312a0vnha65hzs45b6z6vpq-sjson-0.2.1.tar.gz
From https://dustycloud.org/misc/sjson-0.2.1.tar.gz...
 ....1.tar.gz  90KiB                757KiB/s 00:00 [####################] 100.0%
sha256 hash mismatch for output path `/gnu/store/59k6bgh00312a0vnha65hzs45b6z6vpq-sjson-0.2.1.tar.gz'
  expected: 0mzmapln79vv10qxaggz9qwcdbag3jnrj19xx8bgkmxss8h03sv3
  actual:   1mzmapln79vv10qxaggz9qwcdbag3jnrj19xx8bgkmxss8h03sv3
@ build-failed /gnu/store/f4a56cwcb45mkd8vgpjprx06rs4vqhws-sjson-0.2.1.tar.gz.drv - 1 sha256 hash mismatch for output path `/gnu/store/59k6bgh00312a0vnha65hzs45b6z6vpq-sjson-0.2.1.tar.gz'
  expected: 0mzmapln79vv10qxaggz9qwcdbag3jnrj19xx8bgkmxss8h03sv3
  actual:   1mzmapln79vv10qxaggz9qwcdbag3jnrj19xx8bgkmxss8h03sv3
cannot build derivation `/gnu/store/l14gbyb0n7wb2j1ff7ixqah93afzjxia-guile-sjson-0.2.1.drv': 1 dependencies couldn't be built
guix build: error: build failed: build of `/gnu/store/l14gbyb0n7wb2j1ff7ixqah93afzjxia-guile-sjson-0.2.1.drv' failed
rekado in ~: 
--8<---------------cut here---------------end--------------->8---

What do you think?

-- 
Ricardo

Re: Reducing default verbosity

[Danny Milosavljevic]

Hi Ricardo,

On Tue, 5 Dec 2017 14:33:01 +0100
Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> wrote:

> * Reduce package downloads from three lines to one line.  The
>   “Downloading URL” line is only important when things go wrong.
> * Reduce precision in percentages.  The extra digit and the period
>   contribute to visual noise.
> * Widen progress bar, so that it changes more quickly.
> 
>   Before:
>   
>     Downloading https://berlin.guixsd.org/nar/gzip/r43434523452345…-zlib-1.2.11…
>      zlib-1.2.11    151KiB      12.6MiB/s  00:00 [###############] 100.0%
> 
>     Downloading https://berlin.guixsd.org/nar/gzip/rlol…-libpng-1.6.29…
>      libpng-1.6.29  377KiB      10.6MiB/s  00:00 [#############  ]  90.0%
> 
>   After:
> 
>     zlib-1.2.11     151KiB  00:00 [##############################] 100%  12.6MiB/s
>     libpng-1.6.29   377KiB  00:00 [#########################     ]  90%  10.6MiB/s
> 
> * Redirect all build output to log files; as we can’t generally estimate
>   progress I’d use a spinner and maybe display the name of the current
>   build phase and the number of build phases that are left.
> 
>   Building openldap-2.4.45 (2/6)  [‐----->-----------------------]
>   Building openldap-2.4.45 (2/6)  [‐-------------->--------------]
>   Building openldap-2.4.45 (3/6)  [‐----------------------->-----]
>   Building openldap-2.4.45 (4/6)  [‐-----------------<-----------]
>   Building openldap-2.4.45 (4/6)  [-------------<----------------]
> 
>   There are some pretty single character unicode spinners here:
>     https://github.com/pmonks/spinner/blob/master/src/spinner/core.clj
> 
> * Remove double messages that are printed by the daemon and the Guix
>   client, e.g. hash mismatches.  Here’s an example of this:
> 
> --8<---------------cut here---------------start------------->8---
> rekado in ~: guix build guile-sjson
> substitute: updating list of substitutes from 'https://mirror.hydra.gnu.org'... 100.0%
> substitute: updating list of substitutes from 'https://mirror.hydra.gnu.org'... 100.0%
> @ build-started /gnu/store/f4a56cwcb45mkd8vgpjprx06rs4vqhws-sjson-0.2.1.tar.gz.drv - x86_64-linux /var/log/guix/drvs/f4//a56cwcb45mkd8vgpjprx06rs4vqhws-sjson-0.2.1.tar.gz.drv.bz2
> 
> Starting download of /gnu/store/59k6bgh00312a0vnha65hzs45b6z6vpq-sjson-0.2.1.tar.gz
> >From https://dustycloud.org/misc/sjson-0.2.1.tar.gz...  
>  ....1.tar.gz  90KiB                757KiB/s 00:00 [####################] 100.0%
> sha256 hash mismatch for output path `/gnu/store/59k6bgh00312a0vnha65hzs45b6z6vpq-sjson-0.2.1.tar.gz'
>   expected: 0mzmapln79vv10qxaggz9qwcdbag3jnrj19xx8bgkmxss8h03sv3
>   actual:   1mzmapln79vv10qxaggz9qwcdbag3jnrj19xx8bgkmxss8h03sv3
> @ build-failed /gnu/store/f4a56cwcb45mkd8vgpjprx06rs4vqhws-sjson-0.2.1.tar.gz.drv - 1 sha256 hash mismatch for output path `/gnu/store/59k6bgh00312a0vnha65hzs45b6z6vpq-sjson-0.2.1.tar.gz'
>   expected: 0mzmapln79vv10qxaggz9qwcdbag3jnrj19xx8bgkmxss8h03sv3
>   actual:   1mzmapln79vv10qxaggz9qwcdbag3jnrj19xx8bgkmxss8h03sv3
> cannot build derivation `/gnu/store/l14gbyb0n7wb2j1ff7ixqah93afzjxia-guile-sjson-0.2.1.drv': 1 dependencies couldn't be built
> guix build: error: build failed: build of `/gnu/store/l14gbyb0n7wb2j1ff7ixqah93afzjxia-guile-sjson-0.2.1.drv' failed
> rekado in ~: 
> --8<---------------cut here---------------end--------------->8---
> 
> What do you think?

That would be very nice.

I think I tried to do something like the log file stuff before.  The build daemon chooses the log filename and only tells guix frontend about it on success or something.  So not sure whether it would be easy to access the build logs on failure.  I forgot the details, it has been months.

Re: Reducing default verbosity

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 932 bytes --]

On Tue, Dec 05, 2017 at 02:33:01PM +0100, Ricardo Wurmus wrote:
> * Redirect all build output to log files; as we can’t generally estimate
>   progress I’d use a spinner and maybe display the name of the current
>   build phase and the number of build phases that are left.
> 
>   Building openldap-2.4.45 (2/6)  [‐----->-----------------------]
>   Building openldap-2.4.45 (2/6)  [‐-------------->--------------]
>   Building openldap-2.4.45 (3/6)  [‐----------------------->-----]
>   Building openldap-2.4.45 (4/6)  [‐-----------------<-----------]
>   Building openldap-2.4.45 (4/6)  [-------------<----------------]
> 
>   There are some pretty single character unicode spinners here:
>     https://github.com/pmonks/spinner/blob/master/src/spinner/core.clj

Overall I like your suggestions.

I'd prefer to keep the full build output as the default of `guix build`,
and maybe `guix environment`.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Reducing default verbosity

[Ludovic Courtès]

Hello!

Danny Milosavljevic <dannym@scratchpost.org> skribis:

> On Tue, 5 Dec 2017 14:33:01 +0100
> Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> wrote:
>
>> * Reduce package downloads from three lines to one line.  The
>>   “Downloading URL” line is only important when things go wrong.
>> * Reduce precision in percentages.  The extra digit and the period
>>   contribute to visual noise.
>> * Widen progress bar, so that it changes more quickly.
>> 
>>   Before:
>>   
>>     Downloading https://berlin.guixsd.org/nar/gzip/r43434523452345…-zlib-1.2.11…
>>      zlib-1.2.11    151KiB      12.6MiB/s  00:00 [###############] 100.0%
>> 
>>     Downloading https://berlin.guixsd.org/nar/gzip/rlol…-libpng-1.6.29…
>>      libpng-1.6.29  377KiB      10.6MiB/s  00:00 [#############  ]  90.0%
>> 
>>   After:
>> 
>>     zlib-1.2.11     151KiB  00:00 [##############################] 100%  12.6MiB/s
>>     libpng-1.6.29   377KiB  00:00 [#########################     ]  90%  10.6MiB/s
>> 
>> * Redirect all build output to log files; as we can’t generally estimate
>>   progress I’d use a spinner and maybe display the name of the current
>>   build phase and the number of build phases that are left.
>> 
>>   Building openldap-2.4.45 (2/6)  [‐----->-----------------------]
>>   Building openldap-2.4.45 (2/6)  [‐-------------->--------------]
>>   Building openldap-2.4.45 (3/6)  [‐----------------------->-----]
>>   Building openldap-2.4.45 (4/6)  [‐-----------------<-----------]
>>   Building openldap-2.4.45 (4/6)  [-------------<----------------]
>> 
>>   There are some pretty single character unicode spinners here:
>>     https://github.com/pmonks/spinner/blob/master/src/spinner/core.clj
>> 
>> * Remove double messages that are printed by the daemon and the Guix
>>   client, e.g. hash mismatches.  Here’s an example of this:
>> 
>> --8<---------------cut here---------------start------------->8---
>> rekado in ~: guix build guile-sjson
>> substitute: updating list of substitutes from 'https://mirror.hydra.gnu.org'... 100.0%
>> substitute: updating list of substitutes from 'https://mirror.hydra.gnu.org'... 100.0%
>> @ build-started /gnu/store/f4a56cwcb45mkd8vgpjprx06rs4vqhws-sjson-0.2.1.tar.gz.drv - x86_64-linux /var/log/guix/drvs/f4//a56cwcb45mkd8vgpjprx06rs4vqhws-sjson-0.2.1.tar.gz.drv.bz2
>> 
>> Starting download of /gnu/store/59k6bgh00312a0vnha65hzs45b6z6vpq-sjson-0.2.1.tar.gz
>> >From https://dustycloud.org/misc/sjson-0.2.1.tar.gz...  
>>  ....1.tar.gz  90KiB                757KiB/s 00:00 [####################] 100.0%
>> sha256 hash mismatch for output path `/gnu/store/59k6bgh00312a0vnha65hzs45b6z6vpq-sjson-0.2.1.tar.gz'
>>   expected: 0mzmapln79vv10qxaggz9qwcdbag3jnrj19xx8bgkmxss8h03sv3
>>   actual:   1mzmapln79vv10qxaggz9qwcdbag3jnrj19xx8bgkmxss8h03sv3
>> @ build-failed /gnu/store/f4a56cwcb45mkd8vgpjprx06rs4vqhws-sjson-0.2.1.tar.gz.drv - 1 sha256 hash mismatch for output path `/gnu/store/59k6bgh00312a0vnha65hzs45b6z6vpq-sjson-0.2.1.tar.gz'
>>   expected: 0mzmapln79vv10qxaggz9qwcdbag3jnrj19xx8bgkmxss8h03sv3
>>   actual:   1mzmapln79vv10qxaggz9qwcdbag3jnrj19xx8bgkmxss8h03sv3
>> cannot build derivation `/gnu/store/l14gbyb0n7wb2j1ff7ixqah93afzjxia-guile-sjson-0.2.1.drv': 1 dependencies couldn't be built
>> guix build: error: build failed: build of `/gnu/store/l14gbyb0n7wb2j1ff7ixqah93afzjxia-guile-sjson-0.2.1.drv' failed
>> rekado in ~: 
>> --8<---------------cut here---------------end--------------->8---
>> 
>> What do you think?
>
> That would be very nice.

+1!

> I think I tried to do something like the log file stuff before.  The build daemon chooses the log filename and only tells guix frontend about it on success or something.  So not sure whether it would be easy to access the build logs on failure.  I forgot the details, it has been months.

The discussion is at
<https://lists.gnu.org/archive/html/guix-devel/2017-05/msg00497.html>.

The ‘wip-ui’ branch contains one patch I proposed back then to “parse”
build logs so the UI can figure out the current state and important
events and display the right thing.  Danny also proposed a spinner
implementation: <https://bugs.gnu.org/27566>.

Probably time to revive these patches and see what can be done!

Ludo’.

Re: Reducing default verbosity

[Ludovic Courtès]

Leo Famulari <leo@famulari.name> skribis:

> I'd prefer to keep the full build output as the default of `guix build`,
> and maybe `guix environment`.

Yes, maybe just ‘guix build’ if you ask me.

Ludo’.

Re: Reducing default verbosity

[Hartmut Goebel]

Am 05.12.2017 um 14:33 schrieb Ricardo Wurmus:
> What do you think?

IMHO this would be a great improvement.

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: Reducing default verbosity

[Hartmut Goebel]

Am 06.12.2017 um 11:21 schrieb Ludovic Courtès:
> Leo Famulari <leo@famulari.name> skribis:
>
>> I'd prefer to keep the full build output as the default of `guix build`,
>> and maybe `guix environment`.
> Yes, maybe just ‘guix build’ if you ask me.

+1 for ‘guix build’ only.

"guix environment" is often used by end-users who typically are not able
to do anything with this data.

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |