Fetching sources using Guix (re: Building a software toolchain that works)

Fetching sources using Guix (re: Building a software toolchain that works)

[Ryan Prior]

[-- Attachment #1: Type: text/plain, Size: 1782 bytes --]

One of the side-threads in "Building a software toolchain that works" was essentially this:

If I fetch sources for a package using Guix, with the intention to make changes and then build and test the software myself, what should we do with any patches & snippets that are part of the Guix package?

There does not seem to be an obvious right answer. One reason is that patches and snippets fall into at least a few use cases:

- The upstream package won't build at all as-is using the environment Guix creates, so we apply a patch to enable a successful build.
- Upstream vendors some sources into their package, which we prefer to excise using a snippet so that we can do our own dependency management
- Upstream includes non-free components, which we remove to build a fully free package
- Upstream includes binaries in their package, which we prefer to snippet out so we can build them ourselves

At present we don't include any semantic information with each patch or snippet to explain why it is needed. Many have helpful comments; some don't.

Would it be feasible or desirable to create a set of "reason" symbols, similar to our "licenses," and attach a reason (or unknown?) to each snippet and patch? Then we can expose meaningful data to the end-user about patches & snippets that are available, enabling an informed choice about whether to apply them when fetching sources.

This could also be useful in our own record-keeping, so that we can track the usage of patches and snippets for different reasons. It would be nice, for example, to see a downward trend in the number of patches for build systems that won't work at all without them, either because we improve the logic in our build steps or because we contribute changes upstream to make software easier to build.

[-- Attachment #2: Type: text/html, Size: 2991 bytes --]

Re: Fetching sources using Guix (re: Building a software toolchain that works)

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 1405 bytes --]

Ryan Prior schreef op za 19-03-2022 om 00:08 [+0000]:
> Would it be feasible or desirable to create a set of "reason"
> symbols, similar to our "licenses," and attach a reason (or unknown?)
> to each snippet and patch? Then we can expose meaningful data to the
> end-user about patches & snippets that are available, enabling an
> informed choice about whether to apply them when fetching sources.
> 
> This could also be useful in our own record-keeping, so that we can
> track the usage of patches and snippets for different reasons. It
> would be nice, for example, to see a downward trend in the number of
> patches for build systems that won't work at all without them, either
> because we improve the logic in our build steps or because we
> contribute changes upstream to make software easier to build.

FWIW, when reviewing, I try to make sure that the reason is documented
in a comment:

  (origin
    (snippet
      #~(begin
          ;; Remove bundled libraries
          (delete-file-recursively "vendor")
          ;; Remove non-free extension
          (delete-file-recursively "foobar"))))

Additionally, patch files should contain a description what they are
for and a link to the upstream record.

So in theory, what you are describing is already being done, except it
is done as comments and not some machine-readable tagging system.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Fetching sources using Guix (re: Building a software toolchain that works)

[Jonathan McHugh]

Hi Ryan,


March 19, 2022 1:08 AM, "Ryan Prior" <rprior@protonmail.com> wrote:

> One of the side-threads in "Building a software toolchain that works" was essentially this:
> 
> If I fetch sources for a package using Guix, with the intention to make changes and then build and
> test the software myself, what should we do with any patches & snippets that are part of the Guix
> package?
> 
> There does not seem to be an obvious right answer. One reason is that patches and snippets fall
> into at least a few use cases:
> 
> * The upstream package won't build at all as-is using the environment Guix creates, so we apply a
> patch to enable a successful build.
> * Upstream vendors some sources into their package, which we prefer to excise using a snippet so
> that we can do our own dependency management
> * Upstream includes non-free components, which we remove to build a fully free package
> * Upstream includes binaries in their package, which we prefer to snippet out so we can build them
> ourselves
> 
> At present we don't include any semantic information with each patch or snippet to explain why it
> is needed. Many have helpful comments; some don't.
> 
> Would it be feasible or desirable to create a set of "reason" symbols, similar to our "licenses,"
> and attach a reason (or unknown?) to each snippet and patch? Then we can expose meaningful data to
> the end-user about patches & snippets that are available, enabling an informed choice about whether
> to apply them when fetching sources.
> 
I suspect the use of 'reason' symbols huge and potentially with large governance ramifications (such as aligning definitions or enforcing behaviour).

FWIW, Ive expermimented heavily with symbols extensively for years - with Qiuy (what I refer to as a recursive-modelling-language).

For example, here is a list of tokens that exist within HOME folder directories:
https://git.sr.ht/~indieterminacy/2q60mqm_qiuy/tree/main/item/ueu-nen/iwi_2022-03-09

These tokens appear everywhere and Im using them as elemental forces to fascilitate knowledge-management at my end.

Of course, Guix is welcome to come up with its own symbols to highlight desire and state.
It probably needs some deep reflection about needs to be effective.

> This could also be useful in our own record-keeping, so that we can track the usage of patches and
> snippets for different reasons. It would be nice, for example, to see a downward trend in the
> number of patches for build systems that won't work at all without them, either because we improve
> the logic in our build steps or because we contribute changes upstream to make software easier to
> build.


====================
Jonathan McHugh
indieterminacy@libre.brussels