gnu: shepherd: patch, snippet or phase

gnu: shepherd: patch, snippet or phase

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 2921 bytes --]

(as implied per zimoun's previous mail (‘FWIW, it would be unfair for
the patch to have the discussion here’), moved to guix-devel)

Leo Famulari schreef op za 05-03-2022 om 16:13 [-0500]:
> On Thu, Mar 03, 2022 at 07:25:22AM +0100, Maxime Devos wrote:
> > Leo Famulari schreef op wo 02-03-2022 om 18:50 [-0500]:
> > > Origin snippets should only be used to remove nonfree things
> > > from the upstream source code. All other changes should use
> > > patch files or a build phase.
> > 
> > Why?  If it's a source code change and it fits an origin snippet,
> > why not an origin snippet?  Why would the source in Guix need to
> match
> > the source upstream?
> 
> `guix build --source` is a tool to provide freely licensed source
> code
> to be used for any purpose, including building on systems besides
> Guix.
> 
> Using the Guix tools, there is no way to access the upstream source
> code
> without applying the snippets. The reason for that is that the origin
> snippet mechanism was introduced specifically to remove non-free
> components without making it easy to reverse the transformation.

It might be introduced for removing non-free components, that doesn't
mean it cannot be used for more.  Also, I don't see the point of ease
of reversing here. It's trivial to reverse the transformation induced
by the snippet: just delete the snippet in a git checkout.

> Compare that to patch files, which are easily reversed,

Removing a patch file by removing it from the 'patches' field is easy,
as easy as removing a snippet.  I assume you meant the additional
condition ‘... using only CLI tools’?

In that case, you have to: (1) run "guix build --source ..." to compute
the patched source (and maybe unpack), then (2) somehow know there's a
patch to revert, (3) clone guix, (4) find the location of the patch in
the git repo, (5) figure out the invocation of 'patch' command, and
possibly (6) repack the depatched source.  

As such, reversing patch files seems rather difficult to me, it seems
much simpler to just delete the patch inside Guix, and then we might as
well have used a snippet originally.

>  and build
> phases, which do not apply to `guix build --source`.
> 
> So, we have to be careful when using snippets, to ensure that the
> result
> of `guix build --source` is useful on any system, not just Guix.
> 
> More info:
> https://guix.gnu.org/manual/en/html_node/Snippets-versus-Phases.html
> 
> Please let me know if these guidelines are still unclear.

I am aware of the guideline of keeping the source usable outside Guix
systems.  AFAICT, in this case, the snippet modifying
Makefile.am/Makefile.in keeps the source usable on non-Guix systems.
In fact, it makes the source _more_ usable, both on Guix and non-Guix,
by working-around a Guile 3.0.5 compiler bug.  So I don't see any
problems here.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: gnu: shepherd: patch, snippet or phase

[zimoun]

Hi Maxime,

On Sat, 05 Mar 2022 at 22:49, Maxime Devos <maximedevos@telenet.be> wrote:
> (as implied per zimoun's previous mail (‘FWIW, it would be unfair for
> the patch to have the discussion here’), moved to guix-devel)

Thanks. :-)

> Leo Famulari schreef op za 05-03-2022 om 16:13 [-0500]:
>> On Thu, Mar 03, 2022 at 07:25:22AM +0100, Maxime Devos wrote:
>> > Leo Famulari schreef op wo 02-03-2022 om 18:50 [-0500]:
>> > > Origin snippets should only be used to remove nonfree things
>> > > from the upstream source code. All other changes should use
>> > > patch files or a build phase.
>> > 
>> > Why?  If it's a source code change and it fits an origin snippet,
>> > why not an origin snippet?  Why would the source in Guix need to
>> match
>> > the source upstream?
>> 
>> `guix build --source` is a tool to provide freely licensed source
>> code
>> to be used for any purpose, including building on systems besides
>> Guix.
>> 
>> Using the Guix tools, there is no way to access the upstream source
>> code
>> without applying the snippets. The reason for that is that the origin
>> snippet mechanism was introduced specifically to remove non-free
>> components without making it easy to reverse the transformation.
>
> It might be introduced for removing non-free components, that doesn't
> mean it cannot be used for more.  Also, I don't see the point of ease
> of reversing here. It's trivial to reverse the transformation induced
> by the snippet: just delete the snippet in a git checkout.

Well, the point is the FSDG [1] frame, I guess.  From my understanding,
when --source had been introduced, it was a countermeasure to be able to
use hybrid source and still be compliant with an interpretation of: «A
free system distribution must not steer users towards obtaining any
nonfree information for practical use, or encourage them to do so.»

Therefore, using Guix tools, e.g., guix build --source, it is not easy
to reverse what ’snippet’ does.

I would not say it is trivial to reverse the transformation because the
user needs to run “guix edit”, then reassemble the URL, then fetch.
Otherwise, yes the user could go to the Guix repo, remove the snippet,
then run “guix shell -D guix”, do somehow “./pre-inst-env guix …”.

Well, I do not consider these steps “trivial”.  And if one user does
that, somehow they really want to obtain nonfree information. :-)



1: <https://www.gnu.org/distros/free-system-distribution-guidelines.en.html>

>> Compare that to patch files, which are easily reversed,
>
> Removing a patch file by removing it from the 'patches' field is easy,
> as easy as removing a snippet.  I assume you meant the additional
> condition ‘... using only CLI tools’?

Yes, somehow.


> I am aware of the guideline of keeping the source usable outside Guix
> systems.  AFAICT, in this case, the snippet modifying
> Makefile.am/Makefile.in keeps the source usable on non-Guix systems.
> In fact, it makes the source _more_ usable, both on Guix and non-Guix,
> by working-around a Guile 3.0.5 compiler bug.  So I don't see any
> problems here.

Well, the question without consensus is what “guix build --source”
should return?

  a) The source of what “guix build” concretely builds?
  b) The source of upstream (modulo the removal of nonfree part)?

The aim is to be as close as possible as b), IMHO.  The exception of
patches could be discussed. :-)


Back to Shepherd, because the question is originally from patch#54216
[2], the initial snippet was turning a flag:

--8<---------------cut here---------------start------------->8---
+         (snippet
+          '(begin
+             ;; Build with -O1 to work around <https://bugs.gnu.org/48368>.
+             (substitute* "Makefile.am"
+               (("compile --target")
+                "compile -O1 --target"))))))
--8<---------------cut here---------------end--------------->8---

Somehow, the snippet could be considered as a “patch“.  And, in the same
time, the upstream source will not compile without this ’-O1’, IIUC.

However, since “we“ are in the same time upstream and downstream, we
could fix that without introducing this kind of snippet.

Last, because the package is for building with Guix, then it seems more
appropriate to have the substitution in the ’arguments’, as v3 [3] is
doing.


2: <http://issues.guix.gnu.org/issue/54216>
3: <https://issues.guix.gnu.org/issue/54216#19>



Cheers,
simon

Re: gnu: shepherd: patch, snippet or phase

[Zhu Zihao]

[-- Attachment #1: Type: text/plain, Size: 5462 bytes --]



IMO, It's cumbersome to add patches in build phase, you have to add a
new phase, and write something like:

```
(invoke "patch" "-p1" ...)
```

So packager will prefer to add it in the `patches` slot of <origin>
struct. I'd like to see if we have some build procedure like
`apply-patches` to help packager reduce the misnomer of `patches` slot.

Or we can add a new keyword argument #:patches-for-build to
gnu-build-system. For example

```
(package
  (name "XXX")
  (source
   (origin
    ...
    (patches (search-patches "...."))))
  (arguments
   (list
   ;; This patch only used when building the package 
   #:patches-for-build (search-patches ".....")
   ...
   )))
```  


zimoun <zimon.toutoune@gmail.com> writes:

> Hi Maxime,
>
> On Sat, 05 Mar 2022 at 22:49, Maxime Devos <maximedevos@telenet.be> wrote:
>> (as implied per zimoun's previous mail (‘FWIW, it would be unfair for
>> the patch to have the discussion here’), moved to guix-devel)
>
> Thanks. :-)
>
>> Leo Famulari schreef op za 05-03-2022 om 16:13 [-0500]:
>>> On Thu, Mar 03, 2022 at 07:25:22AM +0100, Maxime Devos wrote:
>>> > Leo Famulari schreef op wo 02-03-2022 om 18:50 [-0500]:
>>> > > Origin snippets should only be used to remove nonfree things
>>> > > from the upstream source code. All other changes should use
>>> > > patch files or a build phase.
>>> > 
>>> > Why?  If it's a source code change and it fits an origin snippet,
>>> > why not an origin snippet?  Why would the source in Guix need to
>>> match
>>> > the source upstream?
>>> 
>>> `guix build --source` is a tool to provide freely licensed source
>>> code
>>> to be used for any purpose, including building on systems besides
>>> Guix.
>>> 
>>> Using the Guix tools, there is no way to access the upstream source
>>> code
>>> without applying the snippets. The reason for that is that the origin
>>> snippet mechanism was introduced specifically to remove non-free
>>> components without making it easy to reverse the transformation.
>>
>> It might be introduced for removing non-free components, that doesn't
>> mean it cannot be used for more.  Also, I don't see the point of ease
>> of reversing here. It's trivial to reverse the transformation induced
>> by the snippet: just delete the snippet in a git checkout.
>
> Well, the point is the FSDG [1] frame, I guess.  From my understanding,
> when --source had been introduced, it was a countermeasure to be able to
> use hybrid source and still be compliant with an interpretation of: «A
> free system distribution must not steer users towards obtaining any
> nonfree information for practical use, or encourage them to do so.»
>
> Therefore, using Guix tools, e.g., guix build --source, it is not easy
> to reverse what ’snippet’ does.
>
> I would not say it is trivial to reverse the transformation because the
> user needs to run “guix edit”, then reassemble the URL, then fetch.
> Otherwise, yes the user could go to the Guix repo, remove the snippet,
> then run “guix shell -D guix”, do somehow “./pre-inst-env guix …”.
>
> Well, I do not consider these steps “trivial”.  And if one user does
> that, somehow they really want to obtain nonfree information. :-)
>
>
>
> 1: <https://www.gnu.org/distros/free-system-distribution-guidelines.en.html>
>
>>> Compare that to patch files, which are easily reversed,
>>
>> Removing a patch file by removing it from the 'patches' field is easy,
>> as easy as removing a snippet.  I assume you meant the additional
>> condition ‘... using only CLI tools’?
>
> Yes, somehow.
>
>
>> I am aware of the guideline of keeping the source usable outside Guix
>> systems.  AFAICT, in this case, the snippet modifying
>> Makefile.am/Makefile.in keeps the source usable on non-Guix systems.
>> In fact, it makes the source _more_ usable, both on Guix and non-Guix,
>> by working-around a Guile 3.0.5 compiler bug.  So I don't see any
>> problems here.
>
> Well, the question without consensus is what “guix build --source”
> should return?
>
>   a) The source of what “guix build” concretely builds?
>   b) The source of upstream (modulo the removal of nonfree part)?
>
> The aim is to be as close as possible as b), IMHO.  The exception of
> patches could be discussed. :-)
>
>
> Back to Shepherd, because the question is originally from patch#54216
> [2], the initial snippet was turning a flag:
>
> +         (snippet
> +          '(begin
> +             ;; Build with -O1 to work around <https://bugs.gnu.org/48368>.
> +             (substitute* "Makefile.am"
> +               (("compile --target")
> +                "compile -O1 --target"))))))
>
> Somehow, the snippet could be considered as a “patch“.  And, in the same
> time, the upstream source will not compile without this ’-O1’, IIUC.
>
> However, since “we“ are in the same time upstream and downstream, we
> could fix that without introducing this kind of snippet.
>
> Last, because the package is for building with Guix, then it seems more
> appropriate to have the substitution in the ’arguments’, as v3 [3] is
> doing.
>
>
> 2: <http://issues.guix.gnu.org/issue/54216>
> 3: <https://issues.guix.gnu.org/issue/54216#19>
>
>
>
> Cheers,
> simon


-- 
Retrieve my PGP public key:

  gpg --recv-keys D47A9C8B2AE3905B563D9135BE42B352A9F6821F

Zihao

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 255 bytes --]