What’s next?

What’s next?

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 2421 bytes --]

Hello Guix!

I think it’s time to think about what we want to work on next, ideally
before the next release, and ideally said release should be in a couple
of months rather than in 5 months.  Here are some important items I can
think of:

  • Merge the ncurses installer (the ‘wip-installer’) branch.

  • We’re supposed to freeze ‘core-updates’ in a couple of days and we
    have defined a couple of important goals:
    <https://lists.gnu.org/archive/html/guix-devel/2017-04/msg00120.html>.
    I have a vague feeling that the ‘wip-build-systems-gexp’ merge is
    not for this time yet…

  • ‘guix offload’ terrible bug: <https://bugs.gnu.org/26976>.

  • Guile 2.2 compiler terrible issue:
    <https://lists.gnu.org/archive/html/guile-devel/2017-05/msg00033.html>.

  • Adjust the release process so we don’t find ourselves without
    substitutes for the ‘guix’ package, as reported at
    <https://bugs.gnu.org/27032>.

  • Prepare to migrate to the new build farm: the hardware of
    bayfront.guixsd.org has been fixed (it was unreliable until we
    changed its CPUs), so now we need to fix a couple of issues in
    Cuirass and generally improve it so we can, via its HTTP interface,
    add new jobsets and so on.  Of course we’ll have to work
    on that incrementally.

  • Finalize the new bootloader API and support for new bootloaders:
    <https://bugs.gnu.org/26339>.

  • Merge the potluck!  <https://bugs.gnu.org/26645>

I think everything above could pretty much go into a 0.13.1 release not
far away.  And the missing bits, IMO, for a “1.0” label would be:

  • Improve the UI: add colors (yay!), hide build logs by default for
    most commands, improve progress reports, display how much will be
    downloaded, etc.

  • Implement channels: <https://bugs.gnu.org/22629>.  A first step may
    be to fix some of the obvious shortcomings of ‘guix pull’: use (guix
    git) to optimize bandwidth usage, and compute the derivation of the
    new ‘guix’ and use that.

  • Authenticate Git checkouts: <https://bugs.gnu.org/22883>.

What do people think?

I think the key idea is that we should fix all the annoyances and bugs,
many of which seasoned Guix hackers more or less got used to, but which
are nevertheless a serious hindrance when using Guix “normally.”

Ludo’.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: What’s next?

[Ricardo Wurmus]

Ludovic Courtès <ludo@gnu.org> writes:

> Hello Guix!
>
> I think it’s time to think about what we want to work on next, ideally
> before the next release, and ideally said release should be in a couple
> of months rather than in 5 months.  Here are some important items I can
> think of:

[…]

Excellent points!

> I think the key idea is that we should fix all the annoyances and bugs,
> many of which seasoned Guix hackers more or less got used to, but which
> are nevertheless a serious hindrance when using Guix “normally.”

Here are some other annoyances:

* the verbosity of reporting hash mismatches.  You posted a neat little
  change for that some time ago, but I cannot find it any more.

* texlive.  I’m working on splitting the package up.  (An aggressive
  firewall at work put a stop to that work, but I’ll try to resume very
  soon.)

* very explicit but cryptic stack traces.  I think that all errors
  should be reported more nicely.  The error can still be printed (on
  demand?), but Guix probably should not spill its innards too readily.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: What’s next?

[Brendan Tildesley]

Ludovic Courtès 於 2017-05-24 23:11 寫道:
> Hello Guix!
>
> I think it’s time to think about what we want to work on next, ideally
> before the next release, and ideally said release should be in a couple
> of months rather than in 5 months.  Here are some important items I can
> think of:
[...]
> I think the key idea is that we should fix all the annoyances and bugs,
> many of which seasoned Guix hackers more or less got used to, but which
> are nevertheless a serious hindrance when using Guix “normally.”
>
> Ludo’.

One little annoyance I have is that guix takes every opportunity to
update the list of substitutes when guix build, guix package -u, etc...
is run, and fills my terminal with output like:

substitute: updating list of substitutes from
'https://mirror.hydra.gnu.org'... 100.0%
substitute: updating list of substitutes from
'https://mirror.hydra.gnu.org'... 100.0%
substitute: updating list of substitutes from
'https://mirror.hydra.gnu.org'... 100.0%
substitute: updating list of substitutes from
'https://mirror.hydra.gnu.org'... 100.0%
...

It will output this line a different number each time based on some rule
I couldn't guess, and renders the percentage measurement meaningless.
Currently I am running `guix system init config.scm
--substitute-urls="http://10.1.1.61:8080 https://mirror.hydra.gnu.org/"
--fallback' from the 0.13 installer, and it his been outputting hundreds
of these lines for the last half an hour. I'm not sure if it is ever
going to stop, perhaps I've stumbled across a genuine bug? In any case,
the minor annoyance version is something I've noticed for a while.

Re: What’s next?

[Catonano]

[-- Attachment #1: Type: text/plain, Size: 1351 bytes --]

2017-05-24 15:11 GMT+02:00 Ludovic Courtès <ludo@gnu.org>:

> Hello Guix!
>
> I think it’s time to think about what we want to work on next, ideally
>

one little ting I'd love is Janneke's and Jlicht's solution to use "binary"
nodejs packages

That would allow me to play with some projects that depend on nodejs

The problem of evaluating the adherence of the nodejs collection to the
Guix diistribution guidelines could be faced later ( I posted a thread that
I think is relevant in this regard)



>   • ‘guix offload’ terrible bug: <https://bugs.gnu.org/26976>.
>
>   • Guile 2.2 compiler terrible issue:
>     <https://lists.gnu.org/archive/html/guile-devel/2017-05/msg00033.html
> >.
>

terrible bugs are terrible bugs. They should be regarded as priorities


>   • Improve the UI: add colors (yay!), hide build logs by default for
>     most commands, improve progress reports, display how much will be
>     downloaded, etc.
>

This might be regarded as a small thing but to me this is very important
I use to watch Fedora's colorful terminals with longing


> I think the key idea is that we should fix all the annoyances and bugs,
> many of which seasoned Guix hackers more or less got used to, but which
> are nevertheless a serious hindrance when using Guix “normally.”
>

yeah

[-- Attachment #2: Type: text/html, Size: 2401 bytes --]

Re: What’s next?

[Jan Nieuwenhuizen]

Ludovic Courtès writes:

> I think it’s time to think about what we want to work on next, ideally
> before the next release, and ideally said release should be in a couple
> of months rather than in 5 months.  Here are some important items I can
> think of:

Great points!

>  • Implement channels: <https://bugs.gnu.org/22629>.  A first step may
>    be to fix some of the obvious shortcomings of ‘guix pull’: use (guix
>    git) to optimize bandwidth usage, and compute the derivation of the
>    new ‘guix’ and use that.
>
> I think the key idea is that we should fix all the annoyances and bugs,
> many of which seasoned Guix hackers more or less got used to, but which
> are nevertheless a serious hindrance when using Guix “normally.”

+1

A friend of mine is having a second look at Guix (not SD yet) and one of
the most confusing things initially is `guix pull'.  "When/how do I use
that," he asks...and I can only say: I'm not using that...I think we
want this to work--or something like this, we talked about this at
FOSDEM, but AFAIK everyone is using Guix with Git.

He responds with: then *why* is it in the manual.  I have no answer.
Possibly I'm wrong and/or my information is outdated?

Greetings,
janneke

-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Re: What’s next?

[Adonay Felipe Nogueira]

As an average Guix user, I use `guix pull`.

In fact, I avoid `git pull and the pre-inst-env stuff`. I prefer to work
with GUIX_PACKAGE_PATH in order to work only with `guix package` and
`guix build` if I do have the need to install or build something that I
changed.

-- 
- [[https://libreplanet.org/wiki/User:Adfeno]]
- Palestrante e consultor sobre /software/ livre (não confundir com
  gratis).
- "WhatsApp"? Ele não é livre, por isso não uso. Iguais a ele prefiro
  GNU Ring, ou Tox. Quer outras formas de contato? Adicione o vCard
  que está no endereço acima aos teus contatos.
- Pretende me enviar arquivos .doc, .ppt, .cdr, ou .mp3? OK, eu
  aceito, mas não repasso. Entrego apenas em formatos favoráveis ao
  /software/ livre. Favor entrar em contato em caso de dúvida.

Re: What’s next?

[Catonano]

[-- Attachment #1: Type: text/plain, Size: 1166 bytes --]

2017-05-24 18:25 GMT+02:00 Jan Nieuwenhuizen <janneke@gnu.org>:

>
>
> +1
>
> A friend of mine is having a second look at Guix (not SD yet) and one of
> the most confusing things initially is `guix pull'.  "When/how do I use
> that," he asks...and I can only say: I'm not using that...I think we
> want this to work--or something like this, we talked about this at
> FOSDEM, but AFAIK everyone is using Guix with Git.
>
> He responds with: then *why* is it in the manual.  I have no answer.
> Possibly I'm wrong and/or my information is outdated?
>

This is an important point for me too

I realized that everyone is using git and not guix pull just yesterday

I have been using guix pull until today because I didn' t know any better

The manual mentions something that only beginners use and doesn' t mention
something that the majority of the community is using.

Probably in many of the footages in which Ludo appears, the use of git over
guix pull is assumed.

I think this is a problem. It' s unfair to newcomers and it damages Guix as
a project because it makes the learning curve steeper with not so much of a
point why.

I kinda agree with myglc2, on this.

[-- Attachment #2: Type: text/html, Size: 1702 bytes --]

Re: What’s next?

[Ricardo Wurmus]

Catonano <catonano@gmail.com> writes:

> 2017-05-24 18:25 GMT+02:00 Jan Nieuwenhuizen <janneke@gnu.org>:
[…]
>> A friend of mine is having a second look at Guix (not SD yet) and one of
>> the most confusing things initially is `guix pull'.  "When/how do I use
>> that," he asks...and I can only say: I'm not using that...I think we
>> want this to work--or something like this, we talked about this at
>> FOSDEM, but AFAIK everyone is using Guix with Git.
>>
>> He responds with: then *why* is it in the manual.  I have no answer.
>> Possibly I'm wrong and/or my information is outdated?
>>
>
> This is an important point for me too
>
> I realized that everyone is using git and not guix pull just yesterday
[…]
> I think this is a problem. It' s unfair to newcomers and it damages Guix as
> a project because it makes the learning curve steeper with not so much of a
> point why.

There are two reasons why developers use Guix from git:

* it allows them to add new packages and features to Guix itself.  This
  is something “guix pull” doesn’t support well.

* it doesn’t require compiling all of Guix on each update.

The latter is a defect in “guix pull” that we are aware of and working
on.  Progress has already been made in improving “guix pull”, but it
takes time.  It is not by design that “guix pull” is currently less
desirable than working from a git checkout.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: What’s next?

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 2398 bytes --]

On Wed, May 24, 2017 at 06:25:40PM +0200, Jan Nieuwenhuizen wrote:
> A friend of mine is having a second look at Guix (not SD yet) and one of
> the most confusing things initially is `guix pull'.  "When/how do I use
> that," he asks...and I can only say: I'm not using that...I think we
> want this to work--or something like this, we talked about this at
> FOSDEM, but AFAIK everyone is using Guix with Git.

`guix pull` is one of the primary tools of Guix. For those who are new
to Guix, it should be described as a per-user `apt-get update`. That is,
it updates the list of available packages. The finer differences and
extra features are not important for new users to learn at the
beginning.

With the recent commit adding '--fallback' to `guix pull` [0], the main
reason for Guix users who are not Guix developers to resort to Git has
been removed.

So, I use and recommend `guix pull`!

Do you think the manual can be more clear about this? I'd really like to
hear which parts of the manual your friend read. Maybe we need to
rearrange or rewrite some sections.

I think the most immediate problem with `guix pull` is that it doesn't
support Git commit signature verification. So, you end up trusting
different things: basically, a subset of the X.509 PKI vs PGP+SHA1 [1,2].
I think we can fix this while making `guix pull` use (guix git).

Building Guix from Git is the normal way to develop Guix, and it avoids
downloading a Guix tarball from Savannah in the default case, so
developers will learn and use it, but it brings its own pitfalls.

[0]
https://git.savannah.gnu.org/cgit/guix.git/commit/?id=4902d3c4e0376974356481f222583580b49f39e1

[1] `guix pull` verifies the certificate of <git.savannah.gnu.org>
against the Let's Encrypt trust chain *only*.

[2] If I understand correctly, Git commit signatures are of the SHA1
hash, not the actual commit data. So... not great if I'm correct, but it
will get better as Git introduces a new hash function. And SHA1
collisions are rather obvious to detect, at least according the public
research. An attempt at collision detection was added in Git 2.13.0.

> He responds with: then *why* is it in the manual.  I have no answer.
> Possibly I'm wrong and/or my information is outdated?

Since we are all Guix developers, we talk about developing Guix, but not
as much the day-to-day use. So our impressions may not match actual
usage patterns.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: What’s next?

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 415 bytes --]

On Wed, May 24, 2017 at 09:34:14PM +0200, Catonano wrote:
> I think this is a problem. It' s unfair to newcomers and it damages Guix as
> a project because it makes the learning curve steeper with not so much of a
> point why.

It's true, we collectively worked around some problems with `guix pull`
for too long. Let's all eat the dog food when we don't actually need to
work from Git. I promise it tastes okay :)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: What???s next?

[Pjotr Prins]

On Wed, May 24, 2017 at 05:45:39PM -0400, Leo Famulari wrote:
> [1] `guix pull` verifies the certificate of <git.savannah.gnu.org>
> against the Let's Encrypt trust chain *only*.

This brings up another annoyance. Before a first 'git pull' as a
newbie you have to go through a number of steps which are, arguably,
redundant.

I am talking about installing a first key to trust the guix server.
Well, if we have installed guix AND we use guix pull, I think we can
assume the guix server is trusted (by the user). Therefore, that key
should work out of the box (it is what people install from the tree
anyway!). It is a redundant step. Debian also uses keys and works
out of the box.

The other thing is permissions. Sometimes the user profile needs
explicit permission settings. This is not right. I can see it is
useful on a server setup controlled by an administrator, but arguably
it should just work. The administrator can revert on that. So, if
possible, the default should be allowing guix to work once the daemon
runs.

Pj.

Re: What’s next?

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 694 bytes --]

Leo Famulari <leo@famulari.name> writes:

> So, I use and recommend `guix pull`!

I use it too.  Statements by others in this thread that "nobody" uses it
or that "everyone" is using Git are mistaken.

I use Git when I want to hack on Guix.  Otherwise, I use 'guix pull'.
IMO, the biggest problem with 'guix pull' is that there is no easy
rollback.  I can live with long execution times (--fallback is fine, but
it'd be nice if substitutes were available more often), and I can live
with 'guix pull' causing me to get a version of guix that's broken
somehow, but the inability to easily roll back when things go south
makes me hesitant to run 'guix pull' regularly.

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: What’s next?

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 669 bytes --]

On Thu, May 25, 2017 at 07:57:41AM -0700, Chris Marusich wrote:
> I use Git when I want to hack on Guix.  Otherwise, I use 'guix pull'.
> IMO, the biggest problem with 'guix pull' is that there is no easy
> rollback.  I can live with long execution times (--fallback is fine, but
> it'd be nice if substitutes were available more often), and I can live
> with 'guix pull' causing me to get a version of guix that's broken
> somehow, but the inability to easily roll back when things go south
> makes me hesitant to run 'guix pull' regularly.

It's not "easy rollback", but you can at least use it to deploy any Guix
commit, from any source, with the '--url=' argument.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: What’s next?

[Ricardo Wurmus]

Chris Marusich <cmmarusich@gmail.com> writes:

> Leo Famulari <leo@famulari.name> writes:
>
>> So, I use and recommend `guix pull`!
>
> I use it too.  Statements by others in this thread that "nobody" uses it
> or that "everyone" is using Git are mistaken.
>
> I use Git when I want to hack on Guix.  Otherwise, I use 'guix pull'.
> IMO, the biggest problem with 'guix pull' is that there is no easy
> rollback.  I can live with long execution times (--fallback is fine, but
> it'd be nice if substitutes were available more often), and I can live
> with 'guix pull' causing me to get a version of guix that's broken
> somehow, but the inability to easily roll back when things go south
> makes me hesitant to run 'guix pull' regularly.

I believe this can be fixed by adding more links to “.config/guix”,
i.e. before creating “latest” it would create “2017-05-24:08:21:01.123”
and then link from there to “latest”.  On update it would create a new
link “2017-05-25:17:45:45.123” and link that to latest.  Roll back would
be a matter of pointing “2017-05-24:08:21:01.123” to “latest”.

(“latest” should be renamed to “current” to better match the intent in
this case.)

Timestamped names like this are ugly, but that’s what’s at the top of my
head.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: What’s next?

[Adonay Felipe Nogueira]

I think we can try either using:

- ISO dates: [Full year number]-[Two digit month number]-[Two digit day
  number]T[Two digit hour in 24-hour clock]:[Two digit minutes]:[Two digit
  seconds]

- A numeric sufix: Like "latest.1" for the first generation, "latest.2"
  for the second, and so on, and "latest" for the numbered backup being
  considered as current. Thus, the date and time information would be
  kept in the file metadata, not in the name, specially considering that
  names with ":" might be problematic in some file systems.

I of course don't know, from a developer point of view, what is the best
option, but I thought about giving my "personal" two cents on the
matter. :)

-- 
- [[https://libreplanet.org/wiki/User:Adfeno]]
- Palestrante e consultor sobre /software/ livre (não confundir com
  gratis).
- "WhatsApp"? Ele não é livre, por isso não uso. Iguais a ele prefiro
  GNU Ring, ou Tox. Quer outras formas de contato? Adicione o vCard
  que está no endereço acima aos teus contatos.
- Pretende me enviar arquivos .doc, .ppt, .cdr, ou .mp3? OK, eu
  aceito, mas não repasso. Entrego apenas em formatos favoráveis ao
  /software/ livre. Favor entrar em contato em caso de dúvida.

Re: What’s next?

[Ludovic Courtès]

Hello!

Ricardo Wurmus <rekado@elephly.net> skribis:

> Here are some other annoyances:
>
> * the verbosity of reporting hash mismatches.  You posted a neat little
>   change for that some time ago, but I cannot find it any more.

Oh right, see attached.

> * texlive.  I’m working on splitting the package up.  (An aggressive
>   firewall at work put a stop to that work, but I’ll try to resume very
>   soon.)

Good point!  Looking forward to that.

> * very explicit but cryptic stack traces.  I think that all errors
>   should be reported more nicely.  The error can still be printed (on
>   demand?), but Guix probably should not spill its innards too readily.

I agree with this direction, but we’ll have to work on concrete cases
here.  Bug reports of the form “when I do this I get a stack trace
instead of an error message” are welcome!

Thanks,
Ludo’.

Re: What’s next?

[Ludovic Courtès]

Hi Brendan,

Brendan Tildesley <brendan.tildesley@openmailbox.org> skribis:

> One little annoyance I have is that guix takes every opportunity to
> update the list of substitutes when guix build, guix package -u, etc...
> is run, and fills my terminal with output like:
>
> substitute: updating list of substitutes from
> 'https://mirror.hydra.gnu.org'... 100.0%
> substitute: updating list of substitutes from
> 'https://mirror.hydra.gnu.org'... 100.0%
> substitute: updating list of substitutes from
> 'https://mirror.hydra.gnu.org'... 100.0%
> substitute: updating list of substitutes from
> 'https://mirror.hydra.gnu.org'... 100.0%
> ...

I agree, it’s actually a huge annoyance.  It relates to grafts and how
they are implemented; I took a stab at fixing this a while back but the
approach turned out to be (mostly?) misguided:

  https://bugs.gnu.org/22990

Would be worth thinking through it again!

Thanks,
Ludo’.

Re: What’s next?

[Ludovic Courtès]

Hi!

Jan Nieuwenhuizen <janneke@gnu.org> skribis:

> A friend of mine is having a second look at Guix (not SD yet) and one of
> the most confusing things initially is `guix pull'.  "When/how do I use
> that," he asks...and I can only say: I'm not using that...I think we
> want this to work--or something like this, we talked about this at
> FOSDEM, but AFAIK everyone is using Guix with Git.
>
> He responds with: then *why* is it in the manual.  I have no answer.
> Possibly I'm wrong and/or my information is outdated?

I think there’s a lot of (well deserved, as far as its implementation
goes) badmouthing against ‘guix pull’ on #guix ;-), but nevertheless, I
think it’s a useful tool for users who don’t want to manage their own
Guix checkout etc. by themselves.

The manual does mention in several places what it’s for, I think.

Ludo’.

Re: What’s next?

[Ludovic Courtès]

Ricardo Wurmus <rekado@elephly.net> skribis:

> Chris Marusich <cmmarusich@gmail.com> writes:
>
>> Leo Famulari <leo@famulari.name> writes:
>>
>>> So, I use and recommend `guix pull`!
>>
>> I use it too.  Statements by others in this thread that "nobody" uses it
>> or that "everyone" is using Git are mistaken.
>>
>> I use Git when I want to hack on Guix.  Otherwise, I use 'guix pull'.
>> IMO, the biggest problem with 'guix pull' is that there is no easy
>> rollback.  I can live with long execution times (--fallback is fine, but
>> it'd be nice if substitutes were available more often), and I can live
>> with 'guix pull' causing me to get a version of guix that's broken
>> somehow, but the inability to easily roll back when things go south
>> makes me hesitant to run 'guix pull' regularly.
>
> I believe this can be fixed by adding more links to “.config/guix”,
> i.e. before creating “latest” it would create “2017-05-24:08:21:01.123”
> and then link from there to “latest”.  On update it would create a new
> link “2017-05-25:17:45:45.123” and link that to latest.  Roll back would
> be a matter of pointing “2017-05-24:08:21:01.123” to “latest”.

There would be some similarity with profiles.  Should we simply use
profiles, and effectively turn ~/.config/guix/latest into a profile,
with generations etc.?

Food for thought…  :-)

Ludo’.

Re: What???s next?

[Ludovic Courtès]

Pjotr Prins <pjotr.public12@thebird.nl> skribis:

> On Wed, May 24, 2017 at 05:45:39PM -0400, Leo Famulari wrote:
>> [1] `guix pull` verifies the certificate of <git.savannah.gnu.org>
>> against the Let's Encrypt trust chain *only*.
>
> This brings up another annoyance. Before a first 'git pull' as a
> newbie you have to go through a number of steps which are, arguably,
> redundant.

Note that the Let’s Encrypt certificate check by ‘guix pull’ works out
of the box: users don’t need to install ‘nss-certs’, define a bunch of
environment variables, etc.

> I am talking about installing a first key to trust the guix server.
> Well, if we have installed guix AND we use guix pull, I think we can
> assume the guix server is trusted (by the user). Therefore, that key
> should work out of the box (it is what people install from the tree
> anyway!). It is a redundant step. Debian also uses keys and works
> out of the box.

Substitute servers are fundamentally different from servers that provide
Guix packages, which is why it’s treated differently.

On GuixSD, the key of hydra.gnu.org and bayfront.guixsd.org are always
registered by default.  We cannot do that for someone installing Guix on
a foreign distro because that involves creating a file in /etc.

> The other thing is permissions. Sometimes the user profile needs
> explicit permission settings.

What do you mean?

Thanks,
Ludo’.

Re: What’s next?

[Ricardo Wurmus]

Ludovic Courtès <ludo@gnu.org> writes:

> Hello!
>
> Ricardo Wurmus <rekado@elephly.net> skribis:
>
>> Here are some other annoyances:
>>
>> * the verbosity of reporting hash mismatches.  You posted a neat little
>>   change for that some time ago, but I cannot find it any more.
>
> Oh right, see attached.

I think you forgot to attach it.

>> * texlive.  I’m working on splitting the package up.  (An aggressive
>>   firewall at work put a stop to that work, but I’ll try to resume very
>>   soon.)
>
> Good point!  Looking forward to that.

Me too!

>> * very explicit but cryptic stack traces.  I think that all errors
>>   should be reported more nicely.  The error can still be printed (on
>>   demand?), but Guix probably should not spill its innards too readily.
>
> I agree with this direction, but we’ll have to work on concrete cases
> here.  Bug reports of the form “when I do this I get a stack trace
> instead of an error message” are welcome!

Danny submitted bug #27100 for this with two examples.

-- 
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: What's next?

[Pjotr Prins]

On Sat, May 27, 2017 at 12:16:45PM +0200, Ludovic Court??s wrote:
> On GuixSD, the key of hydra.gnu.org and bayfront.guixsd.org are always
> registered by default.  We cannot do that for someone installing Guix on
> a foreign distro because that involves creating a file in /etc.

Many installs are not on GuixSD. Can't we use the key that is stored
in the store itself? If /etc does not exist then use what comes
with the installation.

> > The other thing is permissions. Sometimes the user profile needs
> > explicit permission settings.
> 
> What do you mean?

I encountered that /var/guix/profiles/ permissions need to be set for
every user. But maybe that has changed, or I got it wrong.

Pj.

Re: What???s next?

[Maxim Cournoyer]

Hello,

ludo@gnu.org (Ludovic Courtès) writes:

[...]

> On GuixSD, the key of hydra.gnu.org and bayfront.guixsd.org are always
> registered by default.  We cannot do that for someone installing Guix on
> a foreign distro because that involves creating a file in /etc.

The bayfront key is now registered by default? Does it mean that
bayfront is ready to be part of the `%default-substitute-urls'? It
doesn't seem to be the case currently:

--8<---------------cut here---------------start------------->8---
(define %default-substitute-urls
  ;; Default list of substituters.  This is *not* the list baked in
  ;; 'guix-daemon', but it is used by 'guix-service-type' and and a couple of
  ;; clients ('guix build --log-file' uses it.)
  (map (if (false-if-exception (resolve-interface '(gnutls)))
           (cut string-append "https://" <>)
           (cut string-append "http://" <>))
       '("mirror.hydra.gnu.org")))
--8<---------------cut here---------------end--------------->8---

Maxim

Re: What’s next?

[Maxim Cournoyer]

Hi,

ludo@gnu.org (Ludovic Courtès) writes:

> Hi Brendan,
>
> Brendan Tildesley <brendan.tildesley@openmailbox.org> skribis:
>
>> One little annoyance I have is that guix takes every opportunity to
>> update the list of substitutes when guix build, guix package -u, etc...
>> is run, and fills my terminal with output like:
>>
>> substitute: updating list of substitutes from
>> 'https://mirror.hydra.gnu.org'... 100.0%
>> substitute: updating list of substitutes from
>> 'https://mirror.hydra.gnu.org'... 100.0%
>> substitute: updating list of substitutes from
>> 'https://mirror.hydra.gnu.org'... 100.0%
>> substitute: updating list of substitutes from
>> 'https://mirror.hydra.gnu.org'... 100.0%
>> ...
>
> I agree, it’s actually a huge annoyance.  It relates to grafts and how
> they are implemented; I took a stab at fixing this a while back but the
> approach turned out to be (mostly?) misguided:
>
>   https://bugs.gnu.org/22990
>
> Would be worth thinking through it again!

Maybe we could make a timer variable configurable, so that at least it
doesn't try to refresh this information at *every* guix command? Default
could be at least one hour. I care about my security, but refreshing
this at every command seems pretty wasteful in terms of resources.

Maxim

Re: What’s next?

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 445 bytes --]

Ricardo Wurmus <rekado@elephly.net> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Hello!
>>
>> Ricardo Wurmus <rekado@elephly.net> skribis:
>>
>>> Here are some other annoyances:
>>>
>>> * the verbosity of reporting hash mismatches.  You posted a neat little
>>>   change for that some time ago, but I cannot find it any more.
>>
>> Oh right, see attached.
>
> I think you forgot to attach it.

Oops.  Here we go:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 1380 bytes --]

modified   nix/libstore/build.cc
@@ -2449,8 +2449,11 @@ void DerivationGoal::registerOutputs()
             Hash h2 = recursive ? hashPath(ht, actualPath).first : hashFile(ht, actualPath);
             if (h != h2)
                 throw BuildError(
-                    format("output path `%1%' should have %2% hash `%3%', instead has `%4%'")
-                    % path % i->second.hashAlgo % printHash16or32(h) % printHash16or32(h2));
+                    format("%1% hash mismatch for output path `%2%'\n"
+			   "  expected: %3%\n"
+			   "  actual:   %4%")
+                    % i->second.hashAlgo % path
+		    % printHash16or32(h) % printHash16or32(h2));
         }
 
         /* Get rid of all weird permissions.  This also checks that
@@ -3096,7 +3099,9 @@ void SubstitutionGoal::finished()
             Hash expectedHash = parseHash16or32(hashType, string(expectedHashStr, n + 1));
             Hash actualHash = hashType == htSHA256 ? hash.first : hashPath(hashType, destPath).first;
             if (expectedHash != actualHash)
-                throw SubstError(format("hash mismatch in downloaded path `%1%': expected %2%, got %3%")
+                throw SubstError(format("hash mismatch in downloaded path `%1%'\n"
+					"  expected: %2%\n"
+					"  actual:   %3%")
                     % storePath % printHash(expectedHash) % printHash(actualHash));
         }

[-- Attachment #3: Type: text/plain, Size: 314 bytes --]


Should we apply it?

>> I agree with this direction, but we’ll have to work on concrete cases
>> here.  Bug reports of the form “when I do this I get a stack trace
>> instead of an error message” are welcome!
>
> Danny submitted bug #27100 for this with two examples.

Great.

Thanks,
Ludo’.

Re: What's next?

[Ludovic Courtès]

Pjotr Prins <pjotr.public12@thebird.nl> skribis:

> On Sat, May 27, 2017 at 12:16:45PM +0200, Ludovic Court??s wrote:
>> On GuixSD, the key of hydra.gnu.org and bayfront.guixsd.org are always
>> registered by default.  We cannot do that for someone installing Guix on
>> a foreign distro because that involves creating a file in /etc.
>
> Many installs are not on GuixSD. Can't we use the key that is stored
> in the store itself? If /etc does not exist then use what comes
> with the installation.

The current behavior is to print a warning when /etc/guix/acl (the list
of authorized keys) is empty or nonexistent.

Your suggestion would be to automatically populate it, right?

I’m mildly reluctant to that, because we’d stealthily force every user
into trusting our substitute servers.  OTOH I agree that the current
situation is not optimal.

What do people think?

Ludo’.

Re: What???s next?

[Ricardo Wurmus]

Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:

> Hello,
>
> ludo@gnu.org (Ludovic Courtès) writes:
>
> [...]
>
>> On GuixSD, the key of hydra.gnu.org and bayfront.guixsd.org are always
>> registered by default.  We cannot do that for someone installing Guix on
>> a foreign distro because that involves creating a file in /etc.
>
> The bayfront key is now registered by default? Does it mean that
> bayfront is ready to be part of the `%default-substitute-urls'? It
> doesn't seem to be the case currently:

No, but the mirror may soon be distributing substitutes that have been
created on and signed by bayfront.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: What’s next?

[Ricardo Wurmus]

Ludovic Courtès <ludo@gnu.org> writes:

> Ricardo Wurmus <rekado@elephly.net> skribis:
>
>> Ludovic Courtès <ludo@gnu.org> writes:
>>
>>> Hello!
>>>
>>> Ricardo Wurmus <rekado@elephly.net> skribis:
>>>
>>>> Here are some other annoyances:
>>>>
>>>> * the verbosity of reporting hash mismatches.  You posted a neat little
>>>>   change for that some time ago, but I cannot find it any more.
>>>
>>> Oh right, see attached.
>>
>> I think you forgot to attach it.
>
> Oops.  Here we go:
>
> modified   nix/libstore/build.cc
> @@ -2449,8 +2449,11 @@ void DerivationGoal::registerOutputs()
>              Hash h2 = recursive ? hashPath(ht, actualPath).first : hashFile(ht, actualPath);
>              if (h != h2)
>                  throw BuildError(
> -                    format("output path `%1%' should have %2% hash `%3%', instead has `%4%'")
> -                    % path % i->second.hashAlgo % printHash16or32(h) % printHash16or32(h2));
> +                    format("%1% hash mismatch for output path `%2%'\n"
> +			   "  expected: %3%\n"
> +			   "  actual:   %4%")
> +                    % i->second.hashAlgo % path
> +		    % printHash16or32(h) % printHash16or32(h2));
>          }
>
>          /* Get rid of all weird permissions.  This also checks that
> @@ -3096,7 +3099,9 @@ void SubstitutionGoal::finished()
>              Hash expectedHash = parseHash16or32(hashType, string(expectedHashStr, n + 1));
>              Hash actualHash = hashType == htSHA256 ? hash.first : hashPath(hashType, destPath).first;
>              if (expectedHash != actualHash)
> -                throw SubstError(format("hash mismatch in downloaded path `%1%': expected %2%, got %3%")
> +                throw SubstError(format("hash mismatch in downloaded path `%1%'\n"
> +					"  expected: %2%\n"
> +					"  actual:   %3%")
>                      % storePath % printHash(expectedHash) % printHash(actualHash));
>          }
>
> Should we apply it?

Yes, please.  This looks much better!  Thank you!

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: What's next?

[Roel Janssen]

Ludovic Courtès writes:

> Pjotr Prins <pjotr.public12@thebird.nl> skribis:
>
>> On Sat, May 27, 2017 at 12:16:45PM +0200, Ludovic Court??s wrote:
>>> On GuixSD, the key of hydra.gnu.org and bayfront.guixsd.org are always
>>> registered by default.  We cannot do that for someone installing Guix on
>>> a foreign distro because that involves creating a file in /etc.
>>
>> Many installs are not on GuixSD. Can't we use the key that is stored
>> in the store itself? If /etc does not exist then use what comes
>> with the installation.
>
> The current behavior is to print a warning when /etc/guix/acl (the list
> of authorized keys) is empty or nonexistent.
>
> Your suggestion would be to automatically populate it, right?
>
> I’m mildly reluctant to that, because we’d stealthily force every user
> into trusting our substitute servers.  OTOH I agree that the current
> situation is not optimal.
>
> What do people think?

Maybe we could find a mid-way here by doing the same as Fedora does with
RPMfusion repositories: It asks the user for trusting the signing keys
before enabling the repository.

So in our case it would be something like:
$ guix package -i emacs
A `substitute' is available for this package on
https://mirror.hydra.gnu.org.  This means we can download the binary
output for this package, instead of compiling it from its source code.
Do you want to use this substitute server with key ... for this package,
and for future packages? [y/N]

We need to find the proper wording for this message.  Using this, we can
still let the user decide, but we can make it a lot easier for the user
to make a decision -- a 'yes' or 'no' answer to a question is easier
than a paragraph in the manual with instructions to enable it.

Kind regards,
Roel Janssen

Re: What's next?

[Maxim Cournoyer]

ludo@gnu.org (Ludovic Courtès) writes:

> Pjotr Prins <pjotr.public12@thebird.nl> skribis:
>
>> On Sat, May 27, 2017 at 12:16:45PM +0200, Ludovic Court??s wrote:
>>> On GuixSD, the key of hydra.gnu.org and bayfront.guixsd.org are always
>>> registered by default.  We cannot do that for someone installing Guix on
>>> a foreign distro because that involves creating a file in /etc.
>>
>> Many installs are not on GuixSD. Can't we use the key that is stored
>> in the store itself? If /etc does not exist then use what comes
>> with the installation.
>
> The current behavior is to print a warning when /etc/guix/acl (the list
> of authorized keys) is empty or nonexistent.
>
> Your suggestion would be to automatically populate it, right?
>
> I’m mildly reluctant to that, because we’d stealthily force every user
> into trusting our substitute servers.  OTOH I agree that the current
> situation is not optimal.
>

Maybe there could be a prompt that tells the user the current message
(no keys in /etc/guix/acl) and then asks them if they'd like to register
the default Guix substitute server keys? That'd be a middle ground
solution.

Maxim

Re: What’s next?

[myglc2]

On 05/27/2017 at 12:13 Ludovic Courtès writes:

> Ricardo Wurmus <rekado@elephly.net> skribis:
>
>> Chris Marusich <cmmarusich@gmail.com> writes:
>>
>>> Leo Famulari <leo@famulari.name> writes:
>>>
>>>> So, I use and recommend `guix pull`!
>>>
>>> I use it too.  Statements by others in this thread that "nobody" uses it
>>> or that "everyone" is using Git are mistaken.
>>>
>>> I use Git when I want to hack on Guix.  Otherwise, I use 'guix pull'.
>>> IMO, the biggest problem with 'guix pull' is that there is no easy
>>> rollback.  I can live with long execution times (--fallback is fine, but
>>> it'd be nice if substitutes were available more often), and I can live
>>> with 'guix pull' causing me to get a version of guix that's broken
>>> somehow, but the inability to easily roll back when things go south
>>> makes me hesitant to run 'guix pull' regularly.
>>
>> I believe this can be fixed by adding more links to “.config/guix”,
>> i.e. before creating “latest” it would create “2017-05-24:08:21:01.123”
>> and then link from there to “latest”.  On update it would create a new
>> link “2017-05-25:17:45:45.123” and link that to latest.  Roll back would
>> be a matter of pointing “2017-05-24:08:21:01.123” to “latest”.
>
> There would be some similarity with profiles.  Should we simply use
> profiles, and effectively turn ~/.config/guix/latest into a profile,
> with generations etc.?

+1

Re: What’s next?

[myglc2]

On 05/24/2017 at 21:56 Ricardo Wurmus writes:

> Catonano <catonano@gmail.com> writes:
>
>> 2017-05-24 18:25 GMT+02:00 Jan Nieuwenhuizen <janneke@gnu.org>:
> […]
>>> A friend of mine is having a second look at Guix (not SD yet) and one of
>>> the most confusing things initially is `guix pull'.  "When/how do I use
>>> that," he asks...and I can only say: I'm not using that...I think we
>>> want this to work--or something like this, we talked about this at
>>> FOSDEM, but AFAIK everyone is using Guix with Git.
>>>
>>> He responds with: then *why* is it in the manual.  I have no answer.
>>> Possibly I'm wrong and/or my information is outdated?
>>>
>>
>> This is an important point for me too
>>
>> I realized that everyone is using git and not guix pull just yesterday
> […]
>> I think this is a problem. It' s unfair to newcomers and it damages Guix as
>> a project because it makes the learning curve steeper with not so much of a
>> point why.
>
> There are two reasons why developers use Guix from git:
>
> * it allows them to add new packages and features to Guix itself.  This
>   is something “guix pull” doesn’t support well.
>
> * it doesn’t require compiling all of Guix on each update.

Hmm, that explains how it feels. Why doesn't it just pull the compiled guix files?

Re: What???s next?

[Ludovic Courtès]

Hi,

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

> ludo@gnu.org (Ludovic Courtès) writes:
>
> [...]
>
>> On GuixSD, the key of hydra.gnu.org and bayfront.guixsd.org are always
>> registered by default.  We cannot do that for someone installing Guix on
>> a foreign distro because that involves creating a file in /etc.
>
> The bayfront key is now registered by default?

Yes!

> Does it mean that bayfront is ready to be part of the
> `%default-substitute-urls'?

Not yet!

Currently it only builds for x86_64/i686, it doesn’t offload anywhere,
etc.  But the fact that it’s registered means that we should eventually
be able to transition transparently (for GuixSD users at least).

Ludo’.

Re: What’s next?

[Ludovic Courtès]

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

> Hi,
>
> ludo@gnu.org (Ludovic Courtès) writes:
>
>> Hi Brendan,
>>
>> Brendan Tildesley <brendan.tildesley@openmailbox.org> skribis:
>>
>>> One little annoyance I have is that guix takes every opportunity to
>>> update the list of substitutes when guix build, guix package -u, etc...
>>> is run, and fills my terminal with output like:
>>>
>>> substitute: updating list of substitutes from
>>> 'https://mirror.hydra.gnu.org'... 100.0%
>>> substitute: updating list of substitutes from
>>> 'https://mirror.hydra.gnu.org'... 100.0%
>>> substitute: updating list of substitutes from
>>> 'https://mirror.hydra.gnu.org'... 100.0%
>>> substitute: updating list of substitutes from
>>> 'https://mirror.hydra.gnu.org'... 100.0%
>>> ...
>>
>> I agree, it’s actually a huge annoyance.  It relates to grafts and how
>> they are implemented; I took a stab at fixing this a while back but the
>> approach turned out to be (mostly?) misguided:
>>
>>   https://bugs.gnu.org/22990
>>
>> Would be worth thinking through it again!
>
> Maybe we could make a timer variable configurable, so that at least it
> doesn't try to refresh this information at *every* guix command?

That’s not what’s happening.  When you see repeated lines that go
directly to 100%, that’s because it’s fetching metadata for just a
single package, and does that one by one.

In other cases, it fetches as many as possible at once, and uses HTTP
pipelining to send all these requests to hydra.gnu.org.

Ludo’.

Re: What's next?

[Ludovic Courtès]

Roel Janssen <roel@gnu.org> skribis:

> Ludovic Courtès writes:
>
>> Pjotr Prins <pjotr.public12@thebird.nl> skribis:
>>
>>> On Sat, May 27, 2017 at 12:16:45PM +0200, Ludovic Court??s wrote:
>>>> On GuixSD, the key of hydra.gnu.org and bayfront.guixsd.org are always
>>>> registered by default.  We cannot do that for someone installing Guix on
>>>> a foreign distro because that involves creating a file in /etc.
>>>
>>> Many installs are not on GuixSD. Can't we use the key that is stored
>>> in the store itself? If /etc does not exist then use what comes
>>> with the installation.
>>
>> The current behavior is to print a warning when /etc/guix/acl (the list
>> of authorized keys) is empty or nonexistent.
>>
>> Your suggestion would be to automatically populate it, right?
>>
>> I’m mildly reluctant to that, because we’d stealthily force every user
>> into trusting our substitute servers.  OTOH I agree that the current
>> situation is not optimal.
>>
>> What do people think?
>
> Maybe we could find a mid-way here by doing the same as Fedora does with
> RPMfusion repositories: It asks the user for trusting the signing keys
> before enabling the repository.
>
> So in our case it would be something like:
> $ guix package -i emacs
> A `substitute' is available for this package on
> https://mirror.hydra.gnu.org.  This means we can download the binary
> output for this package, instead of compiling it from its source code.
> Do you want to use this substitute server with key ... for this package,
> and for future packages? [y/N]

It cannot work this way because the decision has to be made by the
sysadmin, not by unprivileged users.

Also, I like that ‘guix package’ is non-interactive.

Ludo’.

Re: What’s next?

[Ludovic Courtès]

Ricardo Wurmus <rekado@elephly.net> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:

[...]

>> Oops.  Here we go:
>>
>> modified   nix/libstore/build.cc
>> @@ -2449,8 +2449,11 @@ void DerivationGoal::registerOutputs()
>>              Hash h2 = recursive ? hashPath(ht, actualPath).first : hashFile(ht, actualPath);
>>              if (h != h2)
>>                  throw BuildError(
>> -                    format("output path `%1%' should have %2% hash `%3%', instead has `%4%'")
>> -                    % path % i->second.hashAlgo % printHash16or32(h) % printHash16or32(h2));
>> +                    format("%1% hash mismatch for output path `%2%'\n"
>> +			   "  expected: %3%\n"
>> +			   "  actual:   %4%")
>> +                    % i->second.hashAlgo % path
>> +		    % printHash16or32(h) % printHash16or32(h2));
>>          }
>>
>>          /* Get rid of all weird permissions.  This also checks that
>> @@ -3096,7 +3099,9 @@ void SubstitutionGoal::finished()
>>              Hash expectedHash = parseHash16or32(hashType, string(expectedHashStr, n + 1));
>>              Hash actualHash = hashType == htSHA256 ? hash.first : hashPath(hashType, destPath).first;
>>              if (expectedHash != actualHash)
>> -                throw SubstError(format("hash mismatch in downloaded path `%1%': expected %2%, got %3%")
>> +                throw SubstError(format("hash mismatch in downloaded path `%1%'\n"
>> +					"  expected: %2%\n"
>> +					"  actual:   %3%")
>>                      % storePath % printHash(expectedHash) % printHash(actualHash));
>>          }
>>
>> Should we apply it?
>
> Yes, please.  This looks much better!  Thank you!

Applied!

Ludo’.

Re: What's next?

[Pjotr Prins]

On Tue, May 30, 2017 at 05:19:09PM +0200, Ludovic Courtes wrote:
> > A `substitute' is available for this package on
> > https://mirror.hydra.gnu.org.  This means we can download the binary
> > output for this package, instead of compiling it from its source code.
> > Do you want to use this substitute server with key ... for this package,
> > and for future packages? [y/N]
> 
> It cannot work this way because the decision has to be made by the
> sysadmin, not by unprivileged users.

The first time guix is run it is by a system administrator (almost
100% certain), typically from a binary installation.

> Also, I like that 'guix package' is non-interactive.

Sure. But it is a step that is not required on GuixSD. The key comes
with the base install of the guix package. 

Can't we just fetch the key from the store? Why does it have to be
/etc? First look in /etc. If that does not exist, fetch from the guix
package itself. It is there. /etc overrides the key in the store.

Now it is a needless manual step. I say, it is an easy thing to
automate. Automate if you can.

Maybe not the highest priority, but if we come with a solution it
should be easy to do.

Pj.

-- 

Re: What’s next?

[Maxim Cournoyer]

Hello Ludovic,

ludo@gnu.org (Ludovic Courtès) writes:

> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
>
>> Hi,
>>
>> ludo@gnu.org (Ludovic Courtès) writes:
>>
>>> Hi Brendan,
>>>
>>> Brendan Tildesley <brendan.tildesley@openmailbox.org> skribis:
>>>
>>>> One little annoyance I have is that guix takes every opportunity to
>>>> update the list of substitutes when guix build, guix package -u, etc...
>>>> is run, and fills my terminal with output like:
>>>>
>>>> substitute: updating list of substitutes from
>>>> 'https://mirror.hydra.gnu.org'... 100.0%
>>>> substitute: updating list of substitutes from
>>>> 'https://mirror.hydra.gnu.org'... 100.0%
>>>> substitute: updating list of substitutes from
>>>> 'https://mirror.hydra.gnu.org'... 100.0%
>>>> substitute: updating list of substitutes from
>>>> 'https://mirror.hydra.gnu.org'... 100.0%
>>>> ...
>>>
>>> I agree, it’s actually a huge annoyance.  It relates to grafts and how
>>> they are implemented; I took a stab at fixing this a while back but the
>>> approach turned out to be (mostly?) misguided:
>>>
>>>   https://bugs.gnu.org/22990
>>>
>>> Would be worth thinking through it again!
>>
>> Maybe we could make a timer variable configurable, so that at least it
>> doesn't try to refresh this information at *every* guix command?
>
> That’s not what’s happening.  When you see repeated lines that go
> directly to 100%, that’s because it’s fetching metadata for just a
> single package, and does that one by one.
>
> In other cases, it fetches as many as possible at once, and uses HTTP
> pipelining to send all these requests to hydra.gnu.org.

My observation was that for the same exact command (I just tried: "guix
environment emacs-el-mock") entered twice:

--8<---------------cut here---------------start------------->8---
$ guix environment emacs-el-mock
substitute: updating list of substitutes from 'https://bayfront.guixsd.org'...   0.0
substitute: updating list of substitutes from 'https://bayfront.guixsd.org'...  33.3
[...] # (downloads a bunch of substitute)
[env]$ exit
$ guix environment emacs-el-mock
substitute: updating list of substitutes from 'https://bayfront.guixsd.org'... 100.0%
substitute: updating list of substitutes from 'https://bayfront.guixsd.org'... 100.0%
substitute: updating list of substitutes from 'https://bayfront.guixsd.org'... 100.0%
[env]$
--8<---------------cut here---------------end--------------->8---

We can see that a connection is made to the substitute server(s) and
data is retrieved each time the command is entered. Why does it need to
refresh the list of substitutes the second time? Everything needed is
already in the store.

Maxim

Re: What’s next?

[Ricardo Wurmus]

Ludovic Courtès <ludo@gnu.org> writes:

> Ricardo Wurmus <rekado@elephly.net> skribis:
>
>> Chris Marusich <cmmarusich@gmail.com> writes:
>>
>>> Leo Famulari <leo@famulari.name> writes:
>>>
>>>> So, I use and recommend `guix pull`!
>>>
>>> I use it too.  Statements by others in this thread that "nobody" uses it
>>> or that "everyone" is using Git are mistaken.
>>>
>>> I use Git when I want to hack on Guix.  Otherwise, I use 'guix pull'.
>>> IMO, the biggest problem with 'guix pull' is that there is no easy
>>> rollback.  I can live with long execution times (--fallback is fine, but
>>> it'd be nice if substitutes were available more often), and I can live
>>> with 'guix pull' causing me to get a version of guix that's broken
>>> somehow, but the inability to easily roll back when things go south
>>> makes me hesitant to run 'guix pull' regularly.
>>
>> I believe this can be fixed by adding more links to “.config/guix”,
>> i.e. before creating “latest” it would create “2017-05-24:08:21:01.123”
>> and then link from there to “latest”.  On update it would create a new
>> link “2017-05-25:17:45:45.123” and link that to latest.  Roll back would
>> be a matter of pointing “2017-05-24:08:21:01.123” to “latest”.
>
> There would be some similarity with profiles.  Should we simply use
> profiles, and effectively turn ~/.config/guix/latest into a profile,
> with generations etc.?

That’s not a bad idea!  It sure beats messing with a single link and it
makes it possible to more easily manage different versions of Guix.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

What’s next?

[Ludovic Courtès]

Hello Guix!

So, now that 1.3.0 is out the door, what’s next?!

Here’s my wish list of things that look achievable within 4 to 6 months
(I hope to help on some of these):

  • Merging Guix Home.

  • Completing and consolidating Disarchive support (see
    <https://issues.guix.gnu.org/47336>): continuously building the
    Disarchive database, making sure it’s replicated or backed up by
    SWH, and having a blog post or two explaining the whole endeavor
    (I’m looking at you, Timothy ;-)).

  • Merging Magali’s ‘guix git log’ work.

  • Merging ‘core-updates’, perhaps with a switch to GCC 10?  Perhaps
    with support for “simplified packages” (getting rid of input
    labels)?  We’ll see how much can go in there, but the sooner the
    better.

  • On ‘core-updates’, merging either the full-source bootstrap or
    the reduced bootstrap on ARM, whichever comes first.  :-)

  • Consolidating the POWER9 and AArch64 ports (more build machines
    behind ci.guix!).

  • Maybe a step towards “early cutoffs” to reduce the amount of
    rebuilding (more on that in a future post).

  • Maybe optimized substitute downloads based on
    <https://lists.gnu.org/archive/html/guix-devel/2021-01/msg00080.html>
    or at the store item level (same as for early cutoffs).

  • Maybe parameterized packages based on
    <https://lists.gnu.org/archive/html/guix-devel/2020-11/msg00312.html>.

What do people think?  What’s your wish list?  What do you feel an urge
to hack on?  :-)

Ludo’.

Re: What’s next?

[Julien Lepiller]

[-- Attachment #1: Type: text/plain, Size: 2106 bytes --]

In the short term I'd like to merge the python optimisations. We discussed it before release and thought it might be nice to merge them before c-u, from a separate branch.

Maybe before next release we'll have our separate repo on savannah for translations (it's currently a repo I manage on framagit). I'd also like to add some automation so I don't have to manually update the pot files :)

Le 15 mai 2021 13:47:19 GMT-04:00, "Ludovic Courtès" <ludo@gnu.org> a écrit :
>Hello Guix!
>
>So, now that 1.3.0 is out the door, what’s next?!
>
>Here’s my wish list of things that look achievable within 4 to 6 months
>(I hope to help on some of these):
>
>  • Merging Guix Home.
>
>  • Completing and consolidating Disarchive support (see
>    <https://issues.guix.gnu.org/47336>): continuously building the
>    Disarchive database, making sure it’s replicated or backed up by
>    SWH, and having a blog post or two explaining the whole endeavor
>    (I’m looking at you, Timothy ;-)).
>
>  • Merging Magali’s ‘guix git log’ work.
>
>  • Merging ‘core-updates’, perhaps with a switch to GCC 10?  Perhaps
>    with support for “simplified packages” (getting rid of input
>    labels)?  We’ll see how much can go in there, but the sooner the
>    better.
>
>  • On ‘core-updates’, merging either the full-source bootstrap or
>    the reduced bootstrap on ARM, whichever comes first.  :-)
>
>  • Consolidating the POWER9 and AArch64 ports (more build machines
>    behind ci.guix!).
>
>  • Maybe a step towards “early cutoffs” to reduce the amount of
>    rebuilding (more on that in a future post).
>
>  • Maybe optimized substitute downloads based on
>  <https://lists.gnu.org/archive/html/guix-devel/2021-01/msg00080.html>
>    or at the store item level (same as for early cutoffs).
>
>  • Maybe parameterized packages based on
> <https://lists.gnu.org/archive/html/guix-devel/2020-11/msg00312.html>.
>
>What do people think?  What’s your wish list?  What do you feel an urge
>to hack on?  :-)
>
>Ludo’.

[-- Attachment #2: Type: text/html, Size: 2643 bytes --]

Re: What’s next?

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 2065 bytes --]

On Sat, May 15, 2021 at 07:47:19PM +0200, Ludovic Courtès wrote:
> Hello Guix!
> 
> So, now that 1.3.0 is out the door, what’s next?!
> 
> Here’s my wish list of things that look achievable within 4 to 6 months
> (I hope to help on some of these):
> 
>   • Merging Guix Home.
> 
>   • Completing and consolidating Disarchive support (see
>     <https://issues.guix.gnu.org/47336>): continuously building the
>     Disarchive database, making sure it’s replicated or backed up by
>     SWH, and having a blog post or two explaining the whole endeavor
>     (I’m looking at you, Timothy ;-)).
> 
>   • Merging Magali’s ‘guix git log’ work.
> 
>   • Merging ‘core-updates’, perhaps with a switch to GCC 10?  Perhaps
>     with support for “simplified packages” (getting rid of input
>     labels)?  We’ll see how much can go in there, but the sooner the
>     better.
> 
>   • On ‘core-updates’, merging either the full-source bootstrap or
>     the reduced bootstrap on ARM, whichever comes first.  :-)
> 
>   • Consolidating the POWER9 and AArch64 ports (more build machines
>     behind ci.guix!).
> 
>   • Maybe a step towards “early cutoffs” to reduce the amount of
>     rebuilding (more on that in a future post).
> 
>   • Maybe optimized substitute downloads based on
>     <https://lists.gnu.org/archive/html/guix-devel/2021-01/msg00080.html>
>     or at the store item level (same as for early cutoffs).
> 
>   • Maybe parameterized packages based on
>     <https://lists.gnu.org/archive/html/guix-devel/2020-11/msg00312.html>.
> 
> What do people think?  What’s your wish list?  What do you feel an urge
> to hack on?  :-)
> 
> Ludo’.
> 

+1 on core-updates

package-transformations applied to the operating-system field of the
os-config.

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: What’s next?

[Maxim Cournoyer]

Hi Ludovic,

Those are all good ideas, but to be brief:

+1 for core-updates & newer GCC.

Some other things I'd like to see that seem achievable in a 4-6 months
time frame:

* Add .deb and .rpm formats for 'guix pack'.
* Better Texlive importer
* Easier way to debug (guix debug package) -> leaves you in an
environment including everything needed to debug (sources, symbols,
GDB).
* Proper support to mount NFS shares at boot
* guix environment --fhs (file hierachy standard)
* Support for Go modules in the go-build-system
* Better tooling (perhaps contribute to fix perf issues in Geiser/Emacs)
* A bootsplash to finally hide our "Error in finalizer: Success" message ;-)

I'll stop there :-).

Maxim

Re: What’s next?

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 584 bytes --]

Hi,

Off the top of my head:

- Optimize the man page database update during profile builds.

- File search.

  (I did work on it a year ago, it's stuck, would need help from people
  familiar with Cuirass.)

- A GUI.

  https://gitlab.com/daym/guix-gui.git seems to be a good starting point.

Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:

> * Add .deb and .rpm formats for 'guix pack'.
[...]
> * guix environment --fhs (file hierachy standard)

Oh my, awesome ideas!!!  (These 2 in particular.)

Cheers!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 511 bytes --]

What’s next?

[Brendan Tildesley]

[-- Attachment #1: Type: text/plain, Size: 3271 bytes --]

Since you asked I'll dump my nebulous wishes here. Sorry that I don't have very
concrete suggestions and solutions, just things I think could be better.  I
should also state that 99% of my thoughts about Guix are through the filter of
"I want to build a Guix based desktop distribution that can be installed on
library, school, and home computers and Just Work better than Microsoft Windows"

1. Improve guix pull and updating reliability.
Updating guix for me is often like this:
$ guix pull
retrieving git objects....
TLS error in stream blah blah blah
$ guix pull
retrieving git objects..34%......... *dies but doesn't return to the prompt*
ctrl-c
$ guix pull
$ guix upgrade
$ sudo guix system reconfigure...
GUI desktop crashes back to TTY mysteriously.

Basically if guix pull can become as reliable as 'guix pull; guix pull; guix
pull; guix pull;, that would be great.  I think guix has many code paths
pertaining to downloading files where it chooses "Give Up" where "Try 2 or 3
times first" might be better.

Often downloads to source tarballs will download at ~30KiB/s. maybe some
axel-like system where both upstream an mirror urls are used might be good. Also
this may be wrong but from memory I think if the hash is wrong on a source
tarball, it will not bother trying to use the guix ci mirror of it, only if it's
404. So people with --no-substitutes can get completely stuck.

2. 'guix system reconfigure' without instantiating the new system until reboot.
I think the fact that reconfigure changes the whole system while its running is
a bit of a party trick, like pulling the table cloth out without any glasses
falling over.  For a Desktop GNU/Linux it kinda just accidentally works much of
the time. I've always thought I'd prefer the default to simply install the new
configuration in to GRUB and tell the user to reboot or add some flag like
--instatiate-now.


3. CLI consistency, e.g.,
$ guix package => nothing
$ guix system =>
guix system: missing command name
Try 'guix system --help' for more information.

$ guix package --list-generations, but
$ guix system list-generations ???\

Also https://issues.guix.gnu.org/47618


4. Make guix simpler and more performant.

Guix is complex. It has features that make it theoretically superior in many
ways, but in practice hasn't reached the reliability of simpler systems.  Every
aspect of Guix seems to take more time, cpu, ram, hd space than say pacman.
It's a bit beyond my skills to figure out how to actually improve these things
though.

$ guix system foobar

takes 1.5-3.0 seconds on SSD, stats 2374 files, with 345 No such file or
directories in order to determine the command doesn't exist and do nothing.
A fast guix could have 'guix time-machine -- environment --ad-hoc emacs --
emacs' run instantly after its been ran once, and some kind of 'guix run'
command could be added.

5. I think Guix should have some simple system to only update to the latest
commit where behemoths like webkit/chromium already have substitutes available
and that should maybe be the default way to update. It's not a big deal if we
are making people build stuff that only takes 30 seconds and uses minimal
ram. Doesn't need to be perfect. Ricardo made some simple scripts that did stuff
like that in the past.


[-- Attachment #2: Type: text/html, Size: 3958 bytes --]

Re: What’s next?

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 775 bytes --]

Ludovic Courtès schreef op za 15-05-2021 om 19:47 [+0200]:
> Hello Guix!
> 
> So, now that 1.3.0 is out the door, what’s next?!
> 
> Here’s my wish list of things that look achievable within 4 to 6 months
> (I hope to help on some of these): [...]

Distributing substitutes over IPFS.

The original patch series dates from 2018
<https://issues.guix.gnu.org/33899>. We now have an IPFS service
(<https://issues.guix.gnu.org/45905>, merged) and a patch to expose the the gateway
and not only the API (<https://issues.guix.gnu.org/48299>, unmerged).
It seems the only part missing is the IPFS substituter (and relevant "guix publish"
code) itself.

The discussion on IPLD and Unixfs is rather esoteric to me, so I can't help
there.

Greetings,
Maxime

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: What’s next?

[Vagrant Cascadian]

[-- Attachment #1: Type: text/plain, Size: 535 bytes --]

On 2021-05-15, Ludovic Courtès wrote:
> So, now that 1.3.0 is out the door, what’s next?!
>
> Here’s my wish list of things that look achievable within 4 to 6 months
> (I hope to help on some of these):

   • Add a "make dist" job to ci.guix.gnu.org to produce prerelease
     source tarballs.

   • guix lint: support for spellchecker or basic grammar
     https://issues.guix.gnu.org/44675
     Let's have fewer spelling/typo/grammar issues end up in the next
     release tarballs! :)

live well,
  vagrant

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: What’s next?

[Svante Signell]

On Sat, 2021-05-15 at 19:47 +0200, Ludovic Courtès wrote:
> Hello Guix!
> 
> So, now that 1.3.0 is out the door, what’s next?!
> 
> Here’s my wish list of things that look achievable within 4 to 6
> months
> (I hope to help on some of these):

What about Hurd?

Re: What’s next?

[Christopher Lemmer Webber]

Pierre Neidhardt writes:

> Hi,
>
> Off the top of my head:
>
> - Optimize the man page database update during profile builds.
>
> - File search.
>
>   (I did work on it a year ago, it's stuck, would need help from people
>   familiar with Cuirass.)

That sounds interesting.  What does it mean?  Would it help me figure
out which commands come from which package?

A feature I remember being cool they had enabled by default on Ubuntu
back in the day was:

  $ crawl-tiles
  Not found.  Maybe "sudo apt-get install crawl-tiles"?

So for us:

  $ crawl-tiles
  Not found.  Maybe "guix package -i crawl-tiles"?

The curious thing about this though... it seems to require knowledge of
the *outputs*.

To invoke this manually, a user could type:

  # Queries curiass or some sort of build system, similar to guix weather
  $ guix search --binary crawl-tiles
  Found in:
   - crawl-tiles

> - A GUI.
>
>   https://gitlab.com/daym/guix-gui.git seems to be a good starting point.

Oh wow!  I'm going to give this a try.

> Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:
>
>> * Add .deb and .rpm formats for 'guix pack'.
> [...]
>> * guix environment --fhs (file hierachy standard)
>
> Oh my, awesome ideas!!!  (These 2 in particular.)
>
> Cheers!

I'll add one that I *should* finish: I should get the setuid stuff
done.  IIRC it's like, two minor tweaks away from being done and
bitrotting... I've been busy...

Maybe I could do it towards the end of the following week?  Would be
good to get the damned thing done... and then finally get postfix in,
too.

Re: What’s next?

[raingloom]

On Sat, 15 May 2021 23:24:41 +0300
Efraim Flashner <efraim@flashner.co.il> wrote:

> package-transformations applied to the operating-system field of the
> os-config.
> 

Gosh, yes please. I keep bumping into this while debugging services.

Also:
* substitutes over bittorrent might be cool.
* split packages more, similar to how Alpine does it. (i said i'd work
  on this and i intend to)
* deduplicate similar but not identical files (if the file system
  allows it)
* better GDB integration
* better initramfs environment, so debugging isn't such a horrible
  experience
* integrate environments with GUIs. if you've used nested rio on Plan
  9, you know what I'm talking about. I already have some utility
  programs for this, but they are pretty basic and only tested on my
  setup.
* accessible installer! even Arch has this now. (also something I want
  to work on eventually)

Re: What’s next?

[Joshua Branson]

Finally finish my endlessh service.

There's a version that works here:  http://issues.guix.gnu.org/39136

But it runs as root and IS NOT containerized.


P.S.  Also there were some really great suggestions in the previous
emails!

--
Joshua Branson (joshuaBPMan in #guix)
Sent from Emacs and Gnus
  https://gnucode.me
  https://video.hardlimit.com/accounts/joshua_branson/video-channels
  https://propernaming.org
  "You can have whatever you want, as long as you help
enough other people get what they want." - Zig Ziglar

Re: What’s next?

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 1295 bytes --]

Hi Chris,

>> - File search.
>>
>>   (I did work on it a year ago, it's stuck, would need help from people
>>   familiar with Cuirass.)
>
> That sounds interesting.  What does it mean?  Would it help me figure
> out which commands come from which package?

Yes, and more ;)

> A feature I remember being cool they had enabled by default on Ubuntu
> back in the day was:
>
>   $ crawl-tiles
>   Not found.  Maybe "sudo apt-get install crawl-tiles"?
>
> So for us:
>
>   $ crawl-tiles
>   Not found.  Maybe "guix package -i crawl-tiles"?

Guix file-search can do that, but it wouldn't be limited to executables,
it would work for any file.

So `guix filesearch myimage.png`
would return a list of matches:

--8<---------------cut here---------------start------------->8---
emacs-myimage-mode:out  share/assets/myimage.png
...
some-game:assets  share/some-game/subfolder/myimage.png
--8<---------------cut here---------------end--------------->8---

It works by crawling the files and maintaining a database.

The work-in-progress on the wip-filesearch branch works for locally
built packages.  It's obviously very limited, the next thing we need to
do is sync the database from substitute servers.

Cheers!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 511 bytes --]

Re: What’s next?

[zimoun]

Hi,

>   • Completing and consolidating Disarchive support (see
>     <https://issues.guix.gnu.org/47336>): continuously building the
>     Disarchive database, making sure it’s replicated or backed up by
>     SWH, and having a blog post or two explaining the whole endeavor
>     (I’m looking at you, Timothy ;-)).

Adding items here:

 - SVN support.  Chunks are there [1] but missing glue.
 - other (Hg, CVS, etc.)
 - coverage (by the Data Service?)

 - guix time -C channels.scm -- help
   where channels.scm describes missing Git repo, fallback to SWH.

   (Corollary, “guix channel” subcommand helping to add, remove, save,
   etc.)

1: <http://issues.guix.gnu.org/43442#11>

>   • Merging Magali’s ‘guix git log’ work.

About reviewing, feedback welcome. :-) From my understanding, some
commits should be rewritten (split or merged).

In the same direction, I add 2 items:

 - guix time-machine --commit=<OID> / guix pull --commit=<OID>
 - guix git tag: for locally tagging a specific commit

On the next-next step, be able to download information (JSON?) from the
Data Service and use it to easy the “navigation” in the history.  


> What do people think?  What’s your wish list?  What do you feel an urge
> to hack on?  :-)

Add another item:

 - Julia importer (I am working on it, slowly! :-))

Cheers,
simon

PS:  I hope that I have not messed up the In-Reply-To. :-)

Re: What’s next?

[Leo Famulari]

On Sun, May 16, 2021 at 06:26:57PM +0200, Svante Signell wrote:
> What about Hurd?

I think the answer will be: what about it? :)

We always welcome Hurd-related work here in Guix.

Re: What’s next?

[Ludovic Courtès]

Hi,

Efraim Flashner <efraim@flashner.co.il> skribis:

> package-transformations applied to the operating-system field of the
> os-config.

Ah, that’s a good one, but possibly tricky!  What would it operate on?
Any package?  Only those showing up in the system profile?

The former is not really possible; the latter is.

Ludo’.

Re: What’s next?

[Ludovic Courtès]

Hi Brendan,

Brendan Tildesley <btild@mailbox.org> skribis:

> Since you asked I'll dump my nebulous wishes here. Sorry that I don't have very
> concrete suggestions and solutions, just things I think could be better.  I
> should also state that 99% of my thoughts about Guix are through the filter of
> "I want to build a Guix based desktop distribution that can be installed on
> library, school, and home computers and Just Work better than Microsoft Windows"

This is great.  I agree that polishing what already exists should remain
a major goal.

> 1. Improve guix pull and updating reliability.
> Updating guix for me is often like this:
> $ guix pull
> retrieving git objects....
> TLS error in stream blah blah blah
> $ guix pull
> retrieving git objects..34%......... *dies but doesn't return to the prompt*
> ctrl-c

I’ve literally never experienced this.  Could you file a bug or two to
bug-guix@gnu.org, with at least the command and all the output?

> $ guix pull
> $ guix upgrade
> $ sudo guix system reconfigure...
> GUI desktop crashes back to TTY mysteriously.

Likewise, please!

> Often downloads to source tarballs will download at ~30KiB/s. maybe some
> axel-like system where both upstream an mirror urls are used might be
> good.

Could you report as many details when that happens?

> Also this may be wrong but from memory I think if the hash is wrong on
> a source tarball, it will not bother trying to use the guix ci mirror
> of it, only if it's 404. So people with --no-substitutes can get
> completely stuck.

True: <https://issues.guix.gnu.org/28659>.

> 2. 'guix system reconfigure' without instantiating the new system until reboot.
> I think the fact that reconfigure changes the whole system while its running is
> a bit of a party trick, like pulling the table cloth out without any glasses
> falling over.  For a Desktop GNU/Linux it kinda just accidentally works much of
> the time. I've always thought I'd prefer the default to simply install the new
> configuration in to GRUB and tell the user to reboot or add some flag like
> --instatiate-now.

I’m satisfied with the current default, but we could have an option to
delay activation until reboot.  Likewise, please file as a “wishlist”
item to bug-guix@gnu.org.

> 3. CLI consistency, e.g.,
> $ guix package => nothing
> $ guix system =>
> guix system: missing command name
> Try 'guix system --help' for more information.
>
> $ guix package --list-generations, but
> $ guix system list-generations ???\
>
> Also https://issues.guix.gnu.org/47618

All good points!

> 4. Make guix simpler and more performant.
>
> Guix is complex. It has features that make it theoretically superior in many
> ways, but in practice hasn't reached the reliability of simpler systems.  Every
> aspect of Guix seems to take more time, cpu, ram, hd space than say pacman.
> It's a bit beyond my skills to figure out how to actually improve these things
> though.
>
> $ guix system foobar
>
> takes 1.5-3.0 seconds on SSD, stats 2374 files, with 345 No such file or
> directories in order to determine the command doesn't exist and do nothing.

Agreed.  There have been improvements over time, such as the ld.so cache
in ‘core-updates’ (not merged yet), but there’s still a long way.

It’s frustrating for all of us, but one way to help is by pinpointing
specific bottlenecks and gathering as much data as possible so we have a
good starting point for optimization work.

> A fast guix could have 'guix time-machine -- environment --ad-hoc emacs --
> emacs' run instantly after its been ran once, and some kind of 'guix run'
> command could be added.

Yeah.

> 5. I think Guix should have some simple system to only update to the latest
> commit where behemoths like webkit/chromium already have substitutes available
> and that should maybe be the default way to update. It's not a big deal if we
> are making people build stuff that only takes 30 seconds and uses minimal
> ram. Doesn't need to be perfect. Ricardo made some simple scripts that did stuff
> like that in the past.

Yes, this is being worked on (you may have seen
‘channel-with-substitutes-available’ in 1.3.0, which is a first step in
that direction.)

Thanks for sharing!  I guess my message here is that all these should be
broken down into individual bug reports/wishes that can be more easily
addressed.  :-)

Ludo’.

Re: What’s next?

[Joshua Branson]

Leo Famulari <leo@famulari.name> writes:

> On Sun, May 16, 2021 at 06:26:57PM +0200, Svante Signell wrote:
>> What about Hurd?
>
> I think the answer will be: what about it? :)
>
> We always welcome Hurd-related work here in Guix.
>

I suppose someone should fix the Hurd vulnerabilities as reported here:

https://lists.gnu.org/archive/html/bug-hurd/2021-05/msg00079.html

I don't think the vulnerabilities have been disclosed yet nor has there
been a fix yet.

--
Joshua Branson (joshuaBPMan in #guix)
Sent from Emacs and Gnus
  https://gnucode.me
  https://video.hardlimit.com/accounts/joshua_branson/video-channels
  https://propernaming.org
  "You can have whatever you want, as long as you help
enough other people get what they want." - Zig Ziglar

Re: What’s next?

[Bone Baboon]

Ludovic Courtès writes:
> So, now that 1.3.0 is out the door, what’s next?!

> What’s your wish list?  What do you feel an urge to hack on?  :-)

There are two improvements on my Guix wish list.

1) Make the core parts of Guix reproducible
** I do not know if this fits into the 4-6 month time frame mentioned.

2) Alternative kernel
** Motivated by 1.
** Longer term beyond 6 months.

1) Make the core parts of Guix reproducible

Many core parts of Guix are not reproducible.  If more core parts of
Guix were reproducible it would benefit all Guix users.

There are several core parts of Guix that are not reproducible
including:

* Linux-libre
  https://issues.guix.gnu.org/24028#2
  Note: I like what the Linux-libre project is doing.
  	This is likely a result of Linux not being reproducible.

* Many guix-*
  https://issues.guix.gnu.org/48487#0

* Guile
  https://issues.guix.gnu.org/48490#0

* nss 3.59 on the master branch
  https://issues.guix.gnu.org/40316#5

* Emacs
  https://issues.guix.gnu.org/35085#7
  Note: A good text editor is important.
        nvi, vim and neovim are reproducible for me.
	Emacs is more than a text editor and that is a part of why it is
        not reproducible. 

2) Alternative kernel

It is important to have a reproducible kernel.  Linux-libre is not
reproducible (see 1 above).  Linux-libre has not been reproducible for
an extended period of time.  Linux-libre not being reproducible was
reported in 2016 <https://issues.guix.gnu.org/24028#0>.

<https://odysee.com/@Lunduke:e/LinuxSucks2021:1> provides an interesting
thought exercise.  What free libre kernel would Guix use if Linux was
no longer a viable option?  I do not agree with all their points. The
point on Linux complexity increasing rapidly (13:29-17:56) is the one I
would be most concerned about.

Both Linux-libre not being reproducible and the idea that Linux might
not be viable in the future highlight the importance (and potential
urgency) of having an alternative free libre kernel that Guix can run
on.

It is great that work is already underway to get Guix to run on the Gnu
Hurd microkernel.

I think the design concept of a microkernel make them  more resistant to
the problem of increasing complexity at the kernel level when compared
to monolithic kernels.  With microkernels the increased complexity is
pushes out to user processes.  This allows the user (or their operating
system) to choose the level of complexity.

<http://www.microkernel.info/> is a listing of microkernel projects.

Re: What’s next?

[Bone Baboon]

Bone Baboon writes:
> 1) Make the core parts of Guix reproducible
>
> Many core parts of Guix are not reproducible.  If more core parts of
> Guix were reproducible it would benefit all Guix users.
>
> There are several core parts of Guix that are not reproducible
> including:
>
> * Linux-libre
>   https://issues.guix.gnu.org/24028#2
>   Note: I like what the Linux-libre project is doing.
>   	This is likely a result of Linux not being reproducible.
>
> * Many guix-*
>   https://issues.guix.gnu.org/48487#0
>
> * Guile
>   https://issues.guix.gnu.org/48490#0
>
> * nss 3.59 on the master branch
>   https://issues.guix.gnu.org/40316#5
>
> * Emacs
>   https://issues.guix.gnu.org/35085#7
>   Note: A good text editor is important.
>         nvi, vim and neovim are reproducible for me.
> 	Emacs is more than a text editor and that is a part of why it is
>         not reproducible. 

I have one addition to this.  mariadb is failing to build from source
because of a failing test suite.  As mariadb is a dependency of
diffoscope this is causing diffoscope's build from source to also fail.
diffoscope is a useful tool for gathering information about why a
package is not reproducible.

`guix graph --path diffoscope mariadb` outputs:

```
diffoscope@174
ffmpeg@4.3.2
sdl2@2.0.12
fcitx@4.2.9.8
extra-cmake-modules@5.70.0
qtbase@5.15.2
mariadb@10.5.8
```

More details on how mariadb is failing to build can be seen here
<https://issues.guix.gnu.org/48151#3>.

Re: What’s next?

[Leo Famulari]

On Mon, May 17, 2021 at 10:35:22PM -0400, Joshua Branson wrote:
> I suppose someone should fix the Hurd vulnerabilities as reported here:
> 
> https://lists.gnu.org/archive/html/bug-hurd/2021-05/msg00079.html
> 
> I don't think the vulnerabilities have been disclosed yet nor has there
> been a fix yet.

That message is the disclosure.

Re: What’s next?

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 869 bytes --]

Ludovic Courtès schreef op za 15-05-2021 om 19:47 [+0200]:
> [...]
> Here’s my wish list of things that look achievable within 4 to 6 months
> (I hope to help on some of these):
> 
>   • [...]
> 
>   • Merging ‘core-updates’, perhaps with a switch to GCC 10?  Perhaps
>     with support for “simplified packages” (getting rid of input
>     labels)?  We’ll see how much can go in there, but the sooner the
>     better.
> 
>   • On ‘core-updates’, merging either the full-source bootstrap or
>     the reduced bootstrap on ARM, whichever comes first.  :-)
> 
>   [...]
> 

While we are talking about ‘core-updates’, could someone have a look
at a patch series I posted exactly a month ago that fixes a class of
cross-compilation bugs? See <https://issues.guix.gnu.org/47869>.
(No replies so far.)

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: What’s next?

[Leo Famulari]

On Sat, May 15, 2021 at 02:08:50PM -0400, Julien Lepiller wrote:
> In the short term I'd like to merge the python optimisations. We discussed it before release and thought it might be nice to merge them before c-u, from a separate branch.

+1

Have you tested them at all yet? If so, and they don't seem to break
anything, maybe they could be squeezed onto the wip-ungrafting branch.

The 'ungrafting' jobset (based on the wip-ungrafting branch) entails a
complete rebuild of Python 2 and 3. If that branch requires another
iteration due to broken builds, and we are confident that the Python
optimisations won't break things, I'd say we could include them.

Otherwise, we could do a python-updates branch that includes these
optimizations, and updates to Python 3.8.10, along with any existing bug
fixes for Python 2. It would be completed more quickly than
core-updates.

Re: What’s next?

[Ludovic Courtès]

Hi,

Bone Baboon <bone.baboon@disroot.org> skribis:

> 1) Make the core parts of Guix reproducible
>
> Many core parts of Guix are not reproducible.  If more core parts of
> Guix were reproducible it would benefit all Guix users.
>
> There are several core parts of Guix that are not reproducible
> including:
>
> * Linux-libre
>   https://issues.guix.gnu.org/24028#2
>   Note: I like what the Linux-libre project is doing.
>   	This is likely a result of Linux not being reproducible.
>
> * Many guix-*
>   https://issues.guix.gnu.org/48487#0
>
> * Guile
>   https://issues.guix.gnu.org/48490#0
>
> * nss 3.59 on the master branch
>   https://issues.guix.gnu.org/40316#5
>
> * Emacs
>   https://issues.guix.gnu.org/35085#7
>   Note: A good text editor is important.
>         nvi, vim and neovim are reproducible for me.
> 	Emacs is more than a text editor and that is a part of why it is
>         not reproducible. 

All these are great (apart from the Guile and guix-* ones, which are
“known” issues).  I think it takes investigation, to pinpoint the source
of non-determinism, and also checking what other distros do, possibly
asking on the <rb-general@lists.reproducible-builds.org>.

> 2) Alternative kernel

[...]

> It is great that work is already underway to get Guix to run on the Gnu
> Hurd microkernel.

Yup!  Help welcome on this front, too.  :-)

> I think the design concept of a microkernel make them  more resistant to
> the problem of increasing complexity at the kernel level when compared
> to monolithic kernels.  With microkernels the increased complexity is
> pushes out to user processes.  This allows the user (or their operating
> system) to choose the level of complexity.
>
> <http://www.microkernel.info/> is a listing of microkernel projects.

Note that once you have a microkernel, you have next to nothing, by
definition.  The Hurd (+ libc) provides everything you need to provide a
POSIX personality.

Thanks,
Ludo’.

Re: What’s next?

[Ludovic Courtès]

Bonjour !

Julien Lepiller <julien@lepiller.eu> skribis:

> In the short term I'd like to merge the python optimisations. We
> discussed it before release and thought it might be nice to merge them
> before c-u, from a separate branch.

That’d be great!

BTW, another thing that’d be nice it to use Guile-Netlink to provide a
more capable static-networking service.

> Maybe before next release we'll have our separate repo on savannah for
> translations (it's currently a repo I manage on framagit). I'd also
> like to add some automation so I don't have to manually update the pot
> files :)

+1  :-)

Ludo’.

Re: What’s next?

[Julien Lepiller]

[-- Attachment #1: Type: text/plain, Size: 1296 bytes --]

I've rebuilt only up to python 3, and a handful pytgon packages, not a lot more. I'm very confident for the one that enables optimisation, less so for the one that removes files (supposedly useless, like windows binaries and test files).

I'm go for both options, maybe python-update is more wise.

Le 18 mai 2021 15:30:24 GMT-04:00, Leo Famulari <leo@famulari.name> a écrit :
>On Sat, May 15, 2021 at 02:08:50PM -0400, Julien Lepiller wrote:
>> In the short term I'd like to merge the python optimisations. We
>discussed it before release and thought it might be nice to merge them
>before c-u, from a separate branch.
>
>+1
>
>Have you tested them at all yet? If so, and they don't seem to break
>anything, maybe they could be squeezed onto the wip-ungrafting branch.
>
>The 'ungrafting' jobset (based on the wip-ungrafting branch) entails a
>complete rebuild of Python 2 and 3. If that branch requires another
>iteration due to broken builds, and we are confident that the Python
>optimisations won't break things, I'd say we could include them.
>
>Otherwise, we could do a python-updates branch that includes these
>optimizations, and updates to Python 3.8.10, along with any existing
>bug
>fixes for Python 2. It would be completed more quickly than
>core-updates.

[-- Attachment #2: Type: text/html, Size: 1688 bytes --]

Re: What’s next?

[Katherine Cox-Buday]

Ludovic Courtès <ludo@gnu.org> writes:

> BTW, another thing that’d be nice it to use Guile-Netlink to provide a
> more capable static-networking service.

Thank you for the reminder! This is blocking me from standing up a few
edge-routers.

Also, more focus on aarch64-linux substitutes!

-- 
Katherine

Re: What’s next?

[Ricardo Wurmus]

Katherine Cox-Buday <cox.katherine.e@gmail.com> writes:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> BTW, another thing that’d be nice it to use Guile-Netlink to 
>> provide a
>> more capable static-networking service.
>
> Thank you for the reminder! This is blocking me from standing up 
> a few
> edge-routers.
>
> Also, more focus on aarch64-linux substitutes!

We’re currently waiting for three new 16-core Honeycomb LX2 boards 
to arrive Berlin, which should help with aarch64-linux 
substitutes.

-- 
Ricardo

Re: What’s next?

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 852 bytes --]

On Mon, May 17, 2021 at 10:13:10PM +0200, Ludovic Courtès wrote:
> Hi,
> 
> Efraim Flashner <efraim@flashner.co.il> skribis:
> 
> > package-transformations applied to the operating-system field of the
> > os-config.
> 
> Ah, that’s a good one, but possibly tricky!  What would it operate on?
> Any package?  Only those showing up in the system profile?
> 
> The former is not really possible; the latter is.
> 
> Ludo’.
> 

The idea is everything in the system declaration. So the global
packages, the guix-daemon from the guix-daemon service, but not packages
installed using /run/current-system/profile/bin/guix.

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: What’s next?

[raingloom]

On Tue, 18 May 2021 10:05:05 -0400
Leo Famulari <leo@famulari.name> wrote:

> On Mon, May 17, 2021 at 10:35:22PM -0400, Joshua Branson wrote:
> > I suppose someone should fix the Hurd vulnerabilities as reported
> > here:
> > 
> > https://lists.gnu.org/archive/html/bug-hurd/2021-05/msg00079.html
> > 
> > I don't think the vulnerabilities have been disclosed yet nor has
> > there been a fix yet.  
> 
> That message is the disclosure.
> 

Why not put our eggs in a few more baskets with way fewer holes and
more, uh, basket inspectors looking at them, like maybe packaging Minix,
or OpenBSD, or MirageOS, or whatever? I think I stretched that metaphor
but yknow what I mean.
They have seen way more scrutiny than Hurd and also run on more
architectures and while not GPL licensed, AFAIK they are all libre.

Maybe they can't be used in the operating-system-kernel struct field,
but I don't see anything wrong with using Guix to deploy Mirage
unikernel images for instance.
There is even a nascent Scheme unikernel project with Loko Scheme.

Ooor maybe compile some things to WASM and use a WASM+WASI runtime. I
hate webshit but at least there is already tooling and major porting
efforts for WASM.

Re: What’s next?

[Joshua Branson]

Some other cool thoughts:

Package the PHP bits of wordpress.  Then create a wordpress theme that
uses either no or bootstrapable (reproducible?) javascript.  Wordpress
writes some really minimal simple wordpress themes that have minimal
javascript that could be used as a start.  I think there might be a way
that one could then use Emacs to make a really simple blog.  And we
could have a really simple WordPress service.

Or just go crazy a create a competitor to wordpress in GNU guile or
racket.

Package the PHP bits of mediagoblin.  I think this is already done in
one of the WIP branches.  Then create a front-end of media goblin that
uses no or minimal amount bootstrapable (reproducible?) javascript.

One of the guix developers has a WIP git repo web viewer.  And it looks
amazing!  He's got some of his guile code on there. I think it would be
really awesome if savannah's guix git repos could point to that for the
guix repos.  Then we (who am I kidding you, I'm not a developer...yet)
could somehow combine bugs.guix.gnu.org with that guile git repo web
viewer.  The eventual goal could be to create a competitor to sourcehut!

--
Joshua Branson (joshuaBPMan in #guix)
Sent from Emacs and Gnus
  https://gnucode.me
  https://video.hardlimit.com/accounts/joshua_branson/video-channels
  https://propernaming.org
  "You can have whatever you want, as long as you help
enough other people get what they want." - Zig Ziglar

Re: What’s next?

[Ludovic Courtès]

Hi,

Efraim Flashner <efraim@flashner.co.il> skribis:

> On Mon, May 17, 2021 at 10:13:10PM +0200, Ludovic Courtès wrote:
>> Hi,
>> 
>> Efraim Flashner <efraim@flashner.co.il> skribis:
>> 
>> > package-transformations applied to the operating-system field of the
>> > os-config.
>> 
>> Ah, that’s a good one, but possibly tricky!  What would it operate on?
>> Any package?  Only those showing up in the system profile?
>> 
>> The former is not really possible; the latter is.
>> 
>> Ludo’.
>> 
>
> The idea is everything in the system declaration. So the global
> packages, the guix-daemon from the guix-daemon service,

Yeah, that’s not easily possible, in the sense that there’s no place to
“hook” to perform such transforms.  The reason is that references to
packages may be buried down anywhere in gexps or files produced by
services, an obvious example being:

  #~(make-forkexec-constructor
      (list #$(file-append some-package "/bin/xyz")
            …))

The reference to ‘some-package’ here is deep down.

An option would be to have a way to provide ‘lower-object’ a customize
way to lower all the <package> records that it sees, for instance.  But
then, how does that interact with caches, etc.  Tricky!

Ludo’.