Debugging info unavailability

Debugging info unavailability

[Danny Milosavljevic]

Hi,

just now I had to debug a doxygen Segmentation Fault.  I tried to install doxygen:debug but that wasn't available.

I think it would be nice if these outputs were available by default (but not installed by default).

If we wanted to do that, we could just adapt guix/build-system/cmake.scm, guix/build-system/gnu.scm and guix/build-system/glib-or-gtk.scm outputs default to say '("out" "debug") instead of '("out").

Do we want to do that in one of the larger update cycles?

Re: Debugging info unavailability

[Tomas Cech]

On Sun, 23 Apr 2017 02:02:06 +0200,
Danny Milosavljevic wrote:
> 
> Hi,
> 
> just now I had to debug a doxygen Segmentation Fault.  I tried to install doxygen:debug but that wasn't available.
> 
> I think it would be nice if these outputs were available by default (but not installed by default).
> 
> If we wanted to do that, we could just adapt guix/build-system/cmake.scm, guix/build-system/gnu.scm and guix/build-system/glib-or-gtk.scm outputs default to say '("out" "debug") instead of '("out").

This is the feature I wish for years, I'd really appreciate such feature as well.

Best regards,

S_W

Re: Debugging info unavailability

[Ludovic Courtès]

Hi,

Danny Milosavljevic <dannym@scratchpost.org> skribis:

> just now I had to debug a doxygen Segmentation Fault.  I tried to install doxygen:debug but that wasn't available.
>
> I think it would be nice if these outputs were available by default (but not installed by default).

Yeah, on of the reasons this is currently opt-in is disk space on hydra,
as noted in the manual (info "(guix) Installing Debugging Files").

There’s also the fact that packages that do not use the GNU build system
will most likely not produce debugging info out of the box, so adding
“debug” automatically may break many packages.

> If we wanted to do that, we could just adapt guix/build-system/cmake.scm, guix/build-system/gnu.scm and guix/build-system/glib-or-gtk.scm outputs default to say '("out" "debug") instead of '("out").

Rather we should change the default value of the ‘outputs’ field of
<package>.

The problem is that we’d have to add a line like:

  (outputs '("out"))

to all the packages that do not provide debugging symbols (such as
Perl/Python/Ruby packages), which could be a lot of them.  Or we could
provide:

  (define-syntax-rule (package/no-debug fields ...)
    (package
      (outputs '("out"))
      fields ...))

Thoughts?

Ludo’.

Re: Debugging info unavailability

[Maxim Cournoyer]

[-- Attachment #1: Type: text/plain, Size: 1894 bytes --]

Hi,

ludo@gnu.org (Ludovic Courtès) writes:

> Hi,
>
> Danny Milosavljevic <dannym@scratchpost.org> skribis:
>
>> just now I had to debug a doxygen Segmentation Fault.  I tried to install doxygen:debug but that wasn't available.
>>
>> I think it would be nice if these outputs were available by default (but not installed by default).
>
> Yeah, on of the reasons this is currently opt-in is disk space on hydra,
> as noted in the manual (info "(guix) Installing Debugging Files").
>
> There’s also the fact that packages that do not use the GNU build system
> will most likely not produce debugging info out of the box, so adding
> “debug” automatically may break many packages.
>
>> If we wanted to do that, we could just adapt
>> guix/build-system/cmake.scm, guix/build-system/gnu.scm and
>> guix/build-system/glib-or-gtk.scm outputs default to say '("out"
>> "debug") instead of '("out").
>
> Rather we should change the default value of the ‘outputs’ field of
> <package>.
>

Adding the "debug" to the default value of <package> would every package
to now have a debug output; isn't this why Danny suggested to only
change it at the build system level? That way nothing which doesn't have
debugging symbols by default would break or have a useless debug output.

Or, was there something with making the change at that level?

> The problem is that we’d have to add a line like:
>
>   (outputs '("out"))
>
> to all the packages that do not provide debugging symbols (such as
> Perl/Python/Ruby packages), which could be a lot of them.  Or we could
> provide:
>
>   (define-syntax-rule (package/no-debug fields ...)
>     (package
>       (outputs '("out"))
>       fields ...))
>
> Thoughts?
>
> Ludo’.

That seems a more invasive/uglier solution to that hinted by Danny? Or am I
missing something? :)

Thanks,

Maxim

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Debugging info unavailability

[Ludovic Courtès]

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

> ludo@gnu.org (Ludovic Courtès) writes:
>
>> Hi,
>>
>> Danny Milosavljevic <dannym@scratchpost.org> skribis:
>>
>>> just now I had to debug a doxygen Segmentation Fault.  I tried to install doxygen:debug but that wasn't available.
>>>
>>> I think it would be nice if these outputs were available by default (but not installed by default).
>>
>> Yeah, on of the reasons this is currently opt-in is disk space on hydra,
>> as noted in the manual (info "(guix) Installing Debugging Files").
>>
>> There’s also the fact that packages that do not use the GNU build system
>> will most likely not produce debugging info out of the box, so adding
>> “debug” automatically may break many packages.
>>
>>> If we wanted to do that, we could just adapt
>>> guix/build-system/cmake.scm, guix/build-system/gnu.scm and
>>> guix/build-system/glib-or-gtk.scm outputs default to say '("out"
>>> "debug") instead of '("out").
>>
>> Rather we should change the default value of the ‘outputs’ field of
>> <package>.
>>
>
> Adding the "debug" to the default value of <package> would every package
> to now have a debug output; isn't this why Danny suggested to only
> change it at the build system level? That way nothing which doesn't have
> debugging symbols by default would break or have a useless debug output.

Yes, it’s tempting to do it at the build-system level.  However, there
would now be a discrepancy between the actual outputs of the package
derivations and those of the package object: the package object would
declare just one output, but the corresponding derivation would have two
outputs.

I guess bad things would happen if we did that, but maybe someone needs
to try and see exactly what goes wrong.

Ludo’.

Re: Debugging info unavailability

[Maxim Cournoyer]

[-- Attachment #1: Type: text/plain, Size: 2316 bytes --]

ludo@gnu.org (Ludovic Courtès) writes:

> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
>
>> ludo@gnu.org (Ludovic Courtès) writes:
>>
>>> Hi,
>>>
>>> Danny Milosavljevic <dannym@scratchpost.org> skribis:
>>>
>>>> just now I had to debug a doxygen Segmentation Fault.  I tried to install doxygen:debug but that wasn't available.
>>>>
>>>> I think it would be nice if these outputs were available by default (but not installed by default).
>>>
>>> Yeah, on of the reasons this is currently opt-in is disk space on hydra,
>>> as noted in the manual (info "(guix) Installing Debugging Files").
>>>
>>> There’s also the fact that packages that do not use the GNU build system
>>> will most likely not produce debugging info out of the box, so adding
>>> “debug” automatically may break many packages.
>>>
>>>> If we wanted to do that, we could just adapt
>>>> guix/build-system/cmake.scm, guix/build-system/gnu.scm and
>>>> guix/build-system/glib-or-gtk.scm outputs default to say '("out"
>>>> "debug") instead of '("out").
>>>
>>> Rather we should change the default value of the ‘outputs’ field of
>>> <package>.
>>>
>>
>> Adding the "debug" to the default value of <package> would every package
>> to now have a debug output; isn't this why Danny suggested to only
>> change it at the build system level? That way nothing which doesn't have
>> debugging symbols by default would break or have a useless debug output.
>
> Yes, it’s tempting to do it at the build-system level.  However, there
> would now be a discrepancy between the actual outputs of the package
> derivations and those of the package object: the package object would
> declare just one output, but the corresponding derivation would have two
> outputs.
>

Thanks for pointing that! It would be a Bad Thing indeed to introduce a
mismatch between the package definition and the corresponding store
item...

Possibly another Bad Idea, but we could leave things as they are... And
run a script which would rewrite (really, at the package definition
level) the package outputs to include "debug" for every package built
using the gnu/glib-or-gtk build systems? The commit will not be
pretty, that would bring us where we want to be? Being Scheme, that'd be
somewhat easy.

Maxim

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Debugging info unavailability

[Ricardo Wurmus]

Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:

>>> Adding the "debug" to the default value of <package> would every package
>>> to now have a debug output; isn't this why Danny suggested to only
>>> change it at the build system level? That way nothing which doesn't have
>>> debugging symbols by default would break or have a useless debug output.
>>
>> Yes, it’s tempting to do it at the build-system level.  However, there
>> would now be a discrepancy between the actual outputs of the package
>> derivations and those of the package object: the package object would
>> declare just one output, but the corresponding derivation would have two
>> outputs.
>>
>
> Thanks for pointing that! It would be a Bad Thing indeed to introduce a
> mismatch between the package definition and the corresponding store
> item...
>
> Possibly another Bad Idea, but we could leave things as they are... And
> run a script which would rewrite (really, at the package definition
> level) the package outputs to include "debug" for every package built
> using the gnu/glib-or-gtk build systems? The commit will not be
> pretty, that would bring us where we want to be? Being Scheme, that'd be
> somewhat easy.

This sounds better.  I just don’t know if Hydra would have enough space
for all of these additional outputs.

Can we increase storage space on Hydra already or do we need to wait for
bayfront to replace the server in Boston?

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: Debugging info unavailability

[Ludovic Courtès]

Ricardo Wurmus <rekado@elephly.net> skribis:

> Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:
>
>>>> Adding the "debug" to the default value of <package> would every package
>>>> to now have a debug output; isn't this why Danny suggested to only
>>>> change it at the build system level? That way nothing which doesn't have
>>>> debugging symbols by default would break or have a useless debug output.
>>>
>>> Yes, it’s tempting to do it at the build-system level.  However, there
>>> would now be a discrepancy between the actual outputs of the package
>>> derivations and those of the package object: the package object would
>>> declare just one output, but the corresponding derivation would have two
>>> outputs.
>>>
>>
>> Thanks for pointing that! It would be a Bad Thing indeed to introduce a
>> mismatch between the package definition and the corresponding store
>> item...
>>
>> Possibly another Bad Idea, but we could leave things as they are... And
>> run a script which would rewrite (really, at the package definition
>> level) the package outputs to include "debug" for every package built
>> using the gnu/glib-or-gtk build systems? The commit will not be
>> pretty, that would bring us where we want to be? Being Scheme, that'd be
>> somewhat easy.
>
> This sounds better.  I just don’t know if Hydra would have enough space
> for all of these additional outputs.
>
> Can we increase storage space on Hydra already or do we need to wait for
> bayfront to replace the server in Boston?

I don’t think we can have more space easily on hydra.gnu.org.

I’m also unsure how much would be needed.  Currently ‘guix publish’
prepares bakes archives on-demand.  So if you ask for glibc:out, it
returns 404, prepares it, and the next request for glibc:out will
succeed.  But if you ask for glibc:debug, it’s similarly missing
initially.

With this model, ‘guix publish’ gives the impression that not all the
outputs of a given derivation are available at the same time.

We could change ‘guix publish’ to “bake” all the outputs of a derivation
as soon as one if requested—e.g., when you ask for glibc:out, it bakes
not only glibc:out but also glibc:debug.  But then we might have a disk
space issue.

Thoughts?

Ludo’.

Re: Debugging info unavailability

[Maxim Cournoyer]

[-- Attachment #1: Type: text/plain, Size: 3076 bytes --]

ludo@gnu.org (Ludovic Courtès) writes:

> Ricardo Wurmus <rekado@elephly.net> skribis:
>
>> Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:
>>
>>>>> Adding the "debug" to the default value of <package> would every package
>>>>> to now have a debug output; isn't this why Danny suggested to only
>>>>> change it at the build system level? That way nothing which doesn't have
>>>>> debugging symbols by default would break or have a useless debug output.
>>>>
>>>> Yes, it’s tempting to do it at the build-system level.  However, there
>>>> would now be a discrepancy between the actual outputs of the package
>>>> derivations and those of the package object: the package object would
>>>> declare just one output, but the corresponding derivation would have two
>>>> outputs.
>>>>
>>>
>>> Thanks for pointing that! It would be a Bad Thing indeed to introduce a
>>> mismatch between the package definition and the corresponding store
>>> item...
>>>
>>> Possibly another Bad Idea, but we could leave things as they are... And
>>> run a script which would rewrite (really, at the package definition
>>> level) the package outputs to include "debug" for every package built
>>> using the gnu/glib-or-gtk build systems? The commit will not be
>>> pretty, that would bring us where we want to be? Being Scheme, that'd be
>>> somewhat easy.
>>
>> This sounds better.  I just don’t know if Hydra would have enough space
>> for all of these additional outputs.
>>
>> Can we increase storage space on Hydra already or do we need to wait for
>> bayfront to replace the server in Boston?
>
> I don’t think we can have more space easily on hydra.gnu.org.
>
> I’m also unsure how much would be needed.  Currently ‘guix publish’
> prepares bakes archives on-demand.  So if you ask for glibc:out, it
> returns 404, prepares it, and the next request for glibc:out will
> succeed.  But if you ask for glibc:debug, it’s similarly missing
> initially.
>
> With this model, ‘guix publish’ gives the impression that not all the
> outputs of a given derivation are available at the same time.
>
> We could change ‘guix publish’ to “bake” all the outputs of a derivation
> as soon as one if requested—e.g., when you ask for glibc:out, it bakes
> not only glibc:out but also glibc:debug.  But then we might have a disk
> space issue.
>
> Thoughts?
>
> Ludo’.

The ideal situation would be to not be space contrived and to build a cache
everything or at least following some heuristic such as "every package that
was requested at least once in the past month". For someone following
master, I find that the current way substitutes are built is not
aggressive enough, and I find often find myself building the world with
--fallback.

What good is a substitute server if it doesn't hold the stuff I need
*now*? :) On the other side, it really makes me want to look at GNUnet,
which seems like the better long term solution.

I'll investigate the script to add "debug" to our gnu/gtk packages.

Maxim

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Debugging info unavailability

[Ludovic Courtès]

Hi!

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

> The ideal situation would be to not be space contrived and to build a cache
> everything or at least following some heuristic such as "every package that
> was requested at least once in the past month". For someone following
> master, I find that the current way substitutes are built is not
> aggressive enough,

All the packages in ‘master’ are evaluated a couple of times a day
roughly; you can see that at <https://hydra.gnu.org/jobset/gnu/master>.

For x86_64 I think the latency is usually not too bad; sometimes it’s
there’s an increased delay in building the latest packages because
hydra.gnu.org is loaded or something like that.

> and I find often find myself building the world with --fallback.

Maybe that’s because you stumbled upon corrupt items?  I really think
we’re reaching the end of these problems now that we use ‘guix publish
--cache’.

> What good is a substitute server if it doesn't hold the stuff I need
> *now*? :) On the other side, it really makes me want to look at GNUnet,
> which seems like the better long term solution.

Though GNUnet doesn’t solve the fact that one needs a lot of CPU and
storage to build and store all this.  :-)

Ludo’.

Re: Debugging info unavailability

[Ricardo Wurmus]

Ludovic Courtès <ludo@gnu.org> writes:

>> What good is a substitute server if it doesn't hold the stuff I need
>> *now*? :) On the other side, it really makes me want to look at GNUnet,
>> which seems like the better long term solution.
>
> Though GNUnet doesn’t solve the fact that one needs a lot of CPU and
> storage to build and store all this.  :-)

FWIW I’m working on setting up a Guix+Cuirass build farm at the MDC, and
the goal is to eventually make it available to the public.  Initially we
will restrict building to a subset that we use most often (all the
science stuff), but this will result in extra substitutes for packages
lower in the graph.

Ultimately, though, we ought to distribute the workload better, and
“guix publish” already helps with this.

-- 
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: Debugging info unavailability

[Maxim Cournoyer]

Hi!

> Maybe that’s because you stumbled upon corrupt items?  I really think
> we’re reaching the end of these problems now that we use ‘guix publish
> --cache’.
>

That's possible! I've stumbled on a few of those in the past 2 weeks.
It's good to know something's been done about it!

>> What good is a substitute server if it doesn't hold the stuff I need
>> *now*? :) On the other side, it really makes me want to look at GNUnet,
>> which seems like the better long term solution.
>
> Though GNUnet doesn’t solve the fact that one needs a lot of CPU and
> storage to build and store all this.  :-)
>

I think what I meant was "integration of GNUnet with guix publish".
Something which would allow anyone to effortlessly share what's been
built on their machine with the other Guix users. A zero config kind
of thing, with auto discovery of peers and available substitutes.

I haven't researched much about GNUnet yet, but it seems it might be
fit for that purpose.

Maxim

Re: Debugging info unavailability

[Ludovic Courtès]

Ricardo Wurmus <rekado@elephly.net> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>>> What good is a substitute server if it doesn't hold the stuff I need
>>> *now*? :) On the other side, it really makes me want to look at GNUnet,
>>> which seems like the better long term solution.
>>
>> Though GNUnet doesn’t solve the fact that one needs a lot of CPU and
>> storage to build and store all this.  :-)
>
> FWIW I’m working on setting up a Guix+Cuirass build farm at the MDC, and
> the goal is to eventually make it available to the public.  Initially we
> will restrict building to a subset that we use most often (all the
> science stuff), but this will result in extra substitutes for packages
> lower in the graph.

Excellent.  If you’re using a GuixSD config like that of
bayfront.guixsd.org, perhaps your experience will help us streamline
that, such that we can eventually provide, say, a guix-build-farm
service that does the right thing.

> Ultimately, though, we ought to distribute the workload better, and
> “guix publish” already helps with this.

Agreed!

Ludo’.

Distributing substitutes over GNUnet

[Ludovic Courtès]

Hello,

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

> I think what I meant was "integration of GNUnet with guix publish".
> Something which would allow anyone to effortlessly share what's been
> built on their machine with the other Guix users. A zero config kind
> of thing, with auto discovery of peers and available substitutes.

Sure, I agree this is a worthwhile goal.  If you haven’t already, check
out the GSoC by Rémy Birot-Delrue on this topic a couple of years ago:

  https://gnu.org/software/guix/news/gsoc-update.html

Rémy ended up with Guile bindings for GNUnet, along with a
proof-of-concept publish and substitute process using GNUnet:

  https://lists.gnu.org/archive/html/guix-devel/2015-08/msg00455.html

I know ng0 is very interested in this as well.  I’d love to work in this
area!

Ludo’.

Re: Debugging info unavailability

[ng0]

Maxim Cournoyer transcribed 1.0K bytes:
…
> >> What good is a substitute server if it doesn't hold the stuff I need
> >> *now*? :) On the other side, it really makes me want to look at GNUnet,
> >> which seems like the better long term solution.
> >
> > Though GNUnet doesn’t solve the fact that one needs a lot of CPU and
> > storage to build and store all this.  :-)
> >
> 
> I think what I meant was "integration of GNUnet with guix publish".
> Something which would allow anyone to effortlessly share what's been
> built on their machine with the other Guix users. A zero config kind
> of thing, with auto discovery of peers and available substitutes.
> 
> I haven't researched much about GNUnet yet, but it seems it might be
> fit for that purpose.
> 
> Maxim
> 

This has been addressed between 2013? and late 2015, and I'm about to document my own
discussions, thoughts, and roadmap for this (gathered in the last 2 years).
In the sense of freedom of choice, I'd rather make this an opt-in (contrary to what
my own position in discussions was before) so that I can make pragmaOS use this and
those who would like to use it too.
The main roadblocker is 5 weeks - 5 months until a new GNUnet release, but there's
some tasks to work on which can be quickly updated once we have released GNUnet 0.11
or which version number is decided upon.
If you are interested, I can CC you in the message update when I have documented
the ideas (though they are 90% identical to the outcomes of the GSoC discussions
of the past, thought about without knowing it has been discussed before).
My basic idea without going too much into depth (I don't want to search my papers):

- following the ideas of pragmaOS, to first make GNUnet as easy as
  possible to use and configure (the system service I'm working on)
- update the gnunet-guile bindings for HEAD of gnunet but work with 0.10.1 for the
  current version of the service
- write the necessary guix integrations

There's more documented with reasons and ideas beyond just this.
I'm often quoted as the GNUnet expert here in Guix recently (as far as I follow
irc logs), but I would really appreciate if interested people who are passionate
about seeing this feature implemented would join me.
I'm just slow because one person efforts are always slow when the entire project
is not just writing the above listed project but also 3 concepts (blueprint for
how to build a system based on GuixSD, create a base for future systems, package
and write all the things which are required by the blueprint system (which happens
to be focused on GNUnet applications), and last but not least improve the system
by helping upstream).

I know from the meeting december of last year that Ludovic is sceptic about
GNUnet by now to some degree, and if I could decide on releases GNUnet would now
have an -dev or -preview or whatever release. The amount of bugfixes which happened
since 0.10.1 are just too much to keep 0.10.1 around, especially since the compability
no longer works between nodes running 0.10.1 and ones running HEAD.

So far all we can is either take on the last two bugs (I don't know enough C) or
wait until Christian had time to work on them.
-- 
https://pragmatique.xyz
PGP: https://people.pragmatique.xyz/ng0/

Re: Debugging info unavailability

[Maxim Cournoyer]

[-- Attachment #1: Type: text/plain, Size: 3260 bytes --]

Hello ng0,

Sorry for the delayed reply!

ng0 <contact.ng0@cryptolab.net> writes:

> Maxim Cournoyer transcribed 1.0K bytes:
> …
>> >> What good is a substitute server if it doesn't hold the stuff I need
>> >> *now*? :) On the other side, it really makes me want to look at GNUnet,
>> >> which seems like the better long term solution.
>> >
>> > Though GNUnet doesn’t solve the fact that one needs a lot of CPU and
>> > storage to build and store all this.  :-)
>> >
>> 
>> I think what I meant was "integration of GNUnet with guix publish".
>> Something which would allow anyone to effortlessly share what's been
>> built on their machine with the other Guix users. A zero config kind
>> of thing, with auto discovery of peers and available substitutes.
>> 
>> I haven't researched much about GNUnet yet, but it seems it might be
>> fit for that purpose.
>> 
>> Maxim
>> 
>
> This has been addressed between 2013? and late 2015, and I'm about to document my own
> discussions, thoughts, and roadmap for this (gathered in the last 2 years).
> In the sense of freedom of choice, I'd rather make this an opt-in (contrary to what
> my own position in discussions was before) so that I can make pragmaOS use this and
> those who would like to use it too.
> The main roadblocker is 5 weeks - 5 months until a new GNUnet release, but there's
> some tasks to work on which can be quickly updated once we have released GNUnet 0.11
> or which version number is decided upon.
> If you are interested, I can CC you in the message update when I have documented
> the ideas (though they are 90% identical to the outcomes of the GSoC discussions
> of the past, thought about without knowing it has been discussed
> before).

Count me in. I'd like to learn more about GNUnet and any design
ideas/known issues there might be to integrate 'guix publish' with it.

> My basic idea without going too much into depth (I don't want to search my papers):
>
> - following the ideas of pragmaOS, to first make GNUnet as easy as
>   possible to use and configure (the system service I'm working on)

Shouldn't deploying GNUnet on Guix be as easy as adding a service to the
system declaration (and maybe tweeking a few parameters) ?

> - update the gnunet-guile bindings for HEAD of gnunet but work with 0.10.1 for the
>   current version of the service
[...]
> I know from the meeting december of last year that Ludovic is sceptic about
> GNUnet by now to some degree, and if I could decide on releases GNUnet would now
> have an -dev or -preview or whatever release. The amount of bugfixes which happened
> since 0.10.1 are just too much to keep 0.10.1 around, especially since the compability
> no longer works between nodes running 0.10.1 and ones running HEAD.
>

Is backward compatiblity absolutely necessary, given that this version
has its share of problems and is fragmenting the network. I'd rather put
efforts on making GNUnet 'next' working, and well.

Also, why not packaging a GNUnet-next already in Guix? With only two
bugs left to fix, it should already work rather well (maybe better?)
compared to 0.10.1? Guix makes it easy to have multiple versions if we
need -- let's make use of it!

Maxim

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Distributing substitutes over GNUnet

[Maxim Cournoyer]

[-- Attachment #1: Type: text/plain, Size: 1183 bytes --]

Hi,

ludo@gnu.org (Ludovic Courtès) writes:

> Hello,
>
> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
>
>> I think what I meant was "integration of GNUnet with guix publish".
>> Something which would allow anyone to effortlessly share what's been
>> built on their machine with the other Guix users. A zero config kind
>> of thing, with auto discovery of peers and available substitutes.
>
> Sure, I agree this is a worthwhile goal.  If you haven’t already, check
> out the GSoC by Rémy Birot-Delrue on this topic a couple of years ago:
>
>   https://gnu.org/software/guix/news/gsoc-update.html
>
> Rémy ended up with Guile bindings for GNUnet, along with a
> proof-of-concept publish and substitute process using GNUnet:
>
>   https://lists.gnu.org/archive/html/guix-devel/2015-08/msg00455.html
>
> I know ng0 is very interested in this as well.  I’d love to work in this
> area!
>
> Ludo’.

This is a very interesting idea indeed! Thank you for the pointers. I've
taken a quick look at the  code Rémy created in 2015. I wonder if it's
still working. Looks like there are a some tests, so that's good! I'll have to try it!

Maxim

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Debugging info unavailability

[ng0]

Maxim Cournoyer transcribed 4.2K bytes:
> Hello ng0,
> 
> Sorry for the delayed reply!
> 
> ng0 <contact.ng0@cryptolab.net> writes:
> 
> > Maxim Cournoyer transcribed 1.0K bytes:
> > …
> >> >> What good is a substitute server if it doesn't hold the stuff I need
> >> >> *now*? :) On the other side, it really makes me want to look at GNUnet,
> >> >> which seems like the better long term solution.
> >> >
> >> > Though GNUnet doesn’t solve the fact that one needs a lot of CPU and
> >> > storage to build and store all this.  :-)
> >> >
> >> 
> >> I think what I meant was "integration of GNUnet with guix publish".
> >> Something which would allow anyone to effortlessly share what's been
> >> built on their machine with the other Guix users. A zero config kind
> >> of thing, with auto discovery of peers and available substitutes.
> >> 
> >> I haven't researched much about GNUnet yet, but it seems it might be
> >> fit for that purpose.
> >> 
> >> Maxim
> >> 
> >
> > This has been addressed between 2013? and late 2015, and I'm about to document my own
> > discussions, thoughts, and roadmap for this (gathered in the last 2 years).
> > In the sense of freedom of choice, I'd rather make this an opt-in (contrary to what
> > my own position in discussions was before) so that I can make pragmaOS use this and
> > those who would like to use it too.
> > The main roadblocker is 5 weeks - 5 months until a new GNUnet release, but there's
> > some tasks to work on which can be quickly updated once we have released GNUnet 0.11
> > or which version number is decided upon.
> > If you are interested, I can CC you in the message update when I have documented
> > the ideas (though they are 90% identical to the outcomes of the GSoC discussions
> > of the past, thought about without knowing it has been discussed
> > before).
> 
> Count me in. I'd like to learn more about GNUnet and any design
> ideas/known issues there might be to integrate 'guix publish' with it.

This week applications for university started, so I will transfer the
documentation when I have more time. Probably within the next weeks.

> > My basic idea without going too much into depth (I don't want to search my papers):
> >
> > - following the ideas of pragmaOS, to first make GNUnet as easy as
> >   possible to use and configure (the system service I'm working on)
> 
> Shouldn't deploying GNUnet on Guix be as easy as adding a service to the
> system declaration (and maybe tweeking a few parameters) ?

For just the system service, yes. But I want to make the most out of
shepherd and include every possible option which can be later extended
to have example or services documented for configuring the gnunet-vpn, etc
(actually the system-service part is only a small part of improving
the usability).

> > - update the gnunet-guile bindings for HEAD of gnunet but work with 0.10.1 for the
> >   current version of the service
> [...]
> > I know from the meeting december of last year that Ludovic is sceptic about
> > GNUnet by now to some degree, and if I could decide on releases GNUnet would now
> > have an -dev or -preview or whatever release. The amount of bugfixes which happened
> > since 0.10.1 are just too much to keep 0.10.1 around, especially since the compability
> > no longer works between nodes running 0.10.1 and ones running HEAD.
> >
> 
> Is backward compatiblity absolutely necessary, given that this version
> has its share of problems and is fragmenting the network. I'd rather put
> efforts on making GNUnet 'next' working, and well.

There is no backwards compability. This extra step is just done because
in past discussions some of us in Guix insisted on an available release.
Given that we (at GNUnet) now explicitly recommend not to run 0.10.1,
and the only delay is 2 bugs which depend on Christian freeing up time
in the schedule between jobs, it's annoying that we can't use HEAD.

Anyway, for getting the service started it should work like this,
I'm just generally stuck on services and the help on services is
mostly try and run it until it could/should/would work… I don't expect
any help, but there's more help on packages then getting started with
services. It takes much longer.

> Also, why not packaging a GNUnet-next already in Guix? With only two
> bugs left to fix, it should already work rather well (maybe better?)
> compared to 0.10.1? Guix makes it easy to have multiple versions if we
> need -- let's make use of it!

Refer to Ludovic and others for the reasons why we don't have that.
In any case I have many different variants of gnunet in my public
package repository https://git.pragmatique.xyz/ng0-packages/log.html

I could test run the current HEAD with some of my definitions, but
I don't really want to waste my time on discussions about the why
and why not again. If you want it, it can be added very quickly.

> Maxim



-- 
https://pragmatique.xyz
PGP: https://people.pragmatique.xyz/ng0/

Re: Debugging info unavailability

[Ludovic Courtès]

ng0 <ng0@pragmatique.xyz> skribis:

> Maxim Cournoyer transcribed 4.2K bytes:

[...]

>> Also, why not packaging a GNUnet-next already in Guix? With only two
>> bugs left to fix, it should already work rather well (maybe better?)
>> compared to 0.10.1? Guix makes it easy to have multiple versions if we
>> need -- let's make use of it!
>
> Refer to Ludovic and others for the reasons why we don't have that.

Hi!  :-) At the time we discussed it I was reluctant to adding a GNUnet
snapshot as a package because (1) we have the general rule¹ to package
only releases, and (2) when we violate the rule anyway ;-), we need to
be confident that we’re doing the right thing, such as picking the right
commit.

#2 is hard unless you’re also an upstream developer of the software in
question.  For example we wouldn’t want to have Guix users run a version
of GNUnet that uses a protocol compatible neither with the old nor with
the next version.

So I would urge you to lobby on the gnunet-developers list until we get
a shiny new release.  :-)

Ludo’.

¹ https://www.gnu.org/software/guix/manual/html_node/Version-Numbers.html