License compliance when redistributing images built with 'guix system'.

License compliance when redistributing images built with 'guix system'.

[Denis 'GNUtoo' Carikli]

[-- Attachment #1: Type: text/plain, Size: 1818 bytes --]

Hi,

The GPLv2 has this part:
> If distribution of executable or object code is made by offering
> access to copy from a designated place, then offering equivalent 
> access to copy the source code from the same place counts as
> distribution of the source code, even though third parties are not
> compelled to copy the source along with the object code.

So as I understand when redistributing packages (made with guix pack
for instance), 'guix build --sources=transitive
<packages [...]>' can be used to get all the corresponding source code
and publish it in 'the same place'. I could also distribute the Guix
source code being used and the build scripts along the way, and even
generate a package <-> license list with guix package --show just to be
sure (though the licenses are also in the source code).

But for redistributing complete images built with guix system image,
like a VM image for instance, I'm unsure how to do it and I was
wondering if there is a standard way to do that.

The following command seems to be able to tell me which packages were
used in the image:
> guix gc -R "$(guix system build system.scm)"
but then:
(1) I'm unsure if there are all the packages
(2) I need to parse the output and I'm unsure how to separate the
    package name from the version in a reliable way.

Another option could be to export my-os and use something like that in
a scm file that I'd then use 'guix build -L . --sources=transitive' on
after importing (gnu system) and (guix packages):
> (filter package? (operating-system-packages my-os))
but then here too I'm even more afraid of missing packages. The
downside is also that it filters out packages like `(,git "gui") so
using that in the image would not be possible and would need to be
heavily documented.

Denis.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: License compliance when redistributing images built with 'guix system'.

[Simon Tournier]

Hi Denis,

Well, I do not see any reply to this message.  So even very late. :-)

On Thu, 23 May 2024 at 02:09, Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org> wrote:

> The GPLv2 has this part:
>> If distribution of executable or object code is made by offering
>> access to copy from a designated place, then offering equivalent 
>> access to copy the source code from the same place counts as
>> distribution of the source code, even though third parties are not
>> compelled to copy the source along with the object code.
>
> So as I understand when redistributing packages (made with guix pack
> for instance), 'guix build --sources=transitive
> <packages [...]>' can be used to get all the corresponding source code
> and publish it in 'the same place'. I could also distribute the Guix
> source code being used and the build scripts along the way, and even
> generate a package <-> license list with guix package --show just to be
> sure (though the licenses are also in the source code).
>
> But for redistributing complete images built with guix system image,
> like a VM image for instance, I'm unsure how to do it and I was
> wondering if there is a standard way to do that.

Well, I am not sure to get if you speak about the source or about the
binary of this image?

In case you would like to get all the sources required by the system
image, well I am not aware of any easy way.

From your configuration file, you could extract the manifest file which
includes all the packages and based on that you could apply some
“sources=transitive” options.

Cheers,
simon

Re: License compliance when redistributing images built with 'guix system'.

[Denis 'GNUtoo' Carikli]

[-- Attachment #1: Type: text/plain, Size: 1580 bytes --]

On Wed, 17 Jul 2024 19:26:22 +0200
Simon Tournier <zimon.toutoune@gmail.com> wrote:

> Hi Denis,
Hi,

> Well, I do not see any reply to this message.  So even very late. :-)
Thanks a lot.

> > But for redistributing complete images built with guix system image,
> > like a VM image for instance, I'm unsure how to do it and I was
> > wondering if there is a standard way to do that.
> 
> Well, I am not sure to get if you speak about the source or about the
> binary of this image?
The use case is that if I distribute a bootable system image made with
Guix (like a VM image for instance), I'd like an easy way of complying
with all the free software licenses to avoid any legal risks.

Since the image contains many binaries I guess an easy way to comply with
licenses like the GPLv2 could be to provide like a tarball with inside
all the source code corresponding to the binaries inside the VM image.
If there is also extra source code (compilers, etc) it's probably not a
big issue.

As I understand it's easier for distributions like Guix or even more
classical distributions to comply because they typically provide source
code, packages and images from the same servers.

But as a user I'd like to avoid building everything myself, and I'd
like to redistribute some images I make without any legal risks.

> From your configuration file, you could extract the manifest file
> which includes all the packages and based on that you could apply some
> “sources=transitive” options.
Thanks. I think this could be a good option.

Denis.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: License compliance when redistributing images built with 'guix system'.

[Denis 'GNUtoo' Carikli]

[-- Attachment #1: Type: text/plain, Size: 6557 bytes --]

On Wed, 17 Jul 2024 19:26:22 +0200
Simon Tournier <zimon.toutoune@gmail.com> wrote:

> Hi Denis,
Hi,

> Well, I do not see any reply to this message.  So even very late. :-)
Thanks a lot, the answer is very useful.

My work on that was postponed a bit due to more urgent things, but I'm
now back at it. 

This is important to me because I need to distribute some free software
in a Guix image, and one of the thing blocking the publication is the
problem of free software license compliance (the other is documentation
but most of the work is done now).

> In case you would like to get all the sources required by the system
> image, well I am not aware of any easy way.
Thanks for confirming that.

> From your configuration file, you could extract the manifest file
> which includes all the packages and based on that you could apply some
> “sources=transitive” options.
I've tried this approach, and if the manifest file is the one in the
profile, then some software is missing[1].

I've then tried another approach that you probably mentioned in a
recent Guix event in Paris and it looks much more promising but I'm
probably lacking some knowledge to make it work.

I managed to get the image .drv with this command:
> $ guix gc --derivers \
> /gnu/store/lv3sjmhpyjqlzx45im7nyav96sml2k5g-disk-image 
> /gnu/store/qb7inmjc7z5qal98dj9q9xfdcflkxyzf-disk-image.drv

So from that I can easily parse the drv from functions found in Guix
(read-derivation-from-file, derivation-prerequisites, etc) and I can get
much more dependencies but with it I end up listing paths in the
store which are indistinctively binary packages and source code.

Here's my code:
> ;;; Copyright © 2024 Denis Carikli <GNUtoo@cyberdimension.org>
> ;;;
> ;;; This file is free software; you can redistribute it and/or modify
> ;;; it under the terms of the GNU General Public License as published
> ;;; by the Free Software Foundation; either version 3 of the License, or
> ;;; (at your option) any later version.
> ;;;
> ;;; This file is distributed in the hope that it will be useful, but
> ;;; WITHOUT ANY WARRANTY; without even the implied warranty of
> ;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> ;;; GNU General Public License for more details.
> (use-modules (guix derivations))
> 
> (define (endswith str value)
>   (if (< (string-length str) (string-length value))
>       #f
>       (string=?
>        (substring str (- (string-length str) (string-length value))
> (string-length str))
>        value)))
> 
> (let ((drv (read-derivation-from-file (list-ref (program-arguments)
> 1))))
>   (map
>    (lambda (d)
>      (let ((path (car (derivation-input-output-paths d))))
>        (display (string-append path "\n"))))
>    (derivation-prerequisites drv)))

And the result:
> $ guile read-derivations.scm \
> /gnu/store/qb7inmjc7z5qal98dj9q9xfdcflkxyzf-disk-image.drv | \
> grep findutils
> /gnu/store/dxqgxqbvymk5lf1qr31ij22h50yll94h-findutils-4.9.0.tar.zst
> /gnu/store/nmf6js0s9ia3vfli3k2nziaf7s3pj0q4-findutils-4.9.0.tar.xz
> /gnu/store/k09x52pcgw0j8qkvvh5814d87caq2wgn-findutils-4.9.0.tar.xz
> /gnu/store/w4s21yc1m6fpmxcnkcw9zhlkf07mwjgn-findutils-boot0-4.9.0
> /gnu/store/b5c62i99dsfa5j0j3f8g2j1xdi6fih22-findutils-4.9.0
> /gnu/store/w4p913kmnga3xh80vx8vnkri42dcmqi3-findutils-4.9.0

So here I could easily keep only the tarball archives, but then it
breaks with source code coming from git, and I've some that I need to
redistribute in the final image.

So if there is some way to distinguish between source code and
binaries, then we could simply redistribute the source code only and
this would save a lot of space. If not we could still ship it all, but
it'd be harder to justify to the people that will host all that.

References:
===========

[1]How to reproduce the issue:
------------------------------
I used this file:
> ;;; Copyright © 2024 Denis Carikli <GNUtoo@cyberdimension.org>
> ;;; This file is free software; you can redistribute it and/or modify
> ;;; it under the terms of the GNU General Public License as
> ;;; published by the Free Software Foundation; either version 3 of
> ;;; the License, or (at your option) any later version.
> ;;;
> ;;; This file is distributed in the hope that it will be useful, but
> ;;; WITHOUT ANY WARRANTY; without even the implied warranty of
> ;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> ;;; GNU General Public License for more details.
> (use-modules (gnu)
>              (gnu bootloader))
> (operating-system
>   (host-name "minimal")
>   (bootloader (bootloader-configuration
>                 (bootloader grub-bootloader)
>                 (targets '(file-system-label "Guix_image"))
>                 (terminal-outputs '(console))))
>   (file-systems
>    (append (list 
>      (file-system (device (file-system-label "Guix_image"))
>                   (mount-point "/") (type "ext4")))
>    %base-file-systems)))

And I've Guix 56cd9c6735ca1b9bb475291b460d0d2f18352ce1:
> $ guix describe
> Generation 86	Sep 28 2024 14:59:01	(current)
>   guix 56cd9c6
>     repository URL: https://git.savannah.gnu.org/git/guix.git
>     branch: master
>     commit: 56cd9c6735ca1b9bb475291b460d0d2f18352ce1

I then created an image:
> $ guix system image minimal-system.scm
> /gnu/store/lv3sjmhpyjqlzx45im7nyav96sml2k5g-disk-image

I can then mount it:
> $ cp \
> /gnu/store/lv3sjmhpyjqlzx45im7nyav96sml2k5g-disk-image disk.img
> $ chmod +w disk.img
> $ udisksctl loop-setup -f disk.img
> Mapped file disk.img as /dev/loop0.
> $ udisksctl mount -b /dev/loop0p2 
> Mounted /dev/loop0p2 at /media/gnutoo/Guix_image.

And for instance I've the ath9k_htc firmware:
> $ ls -d /media/gnutoo/Guix_image/gnu/store/*ath9k*
> /media/gnutoo/Guix_image/gnu/store/d5xz8k0wq0qmr7c3p90r6njxk3fgabk2-ath9k-htc-ar7010-firmware-1.4.0
> /media/gnutoo/Guix_image/gnu/store/rx6azgdb3ahxv5lpdl35l717r0a1gg1h-ath9k-htc-ar9271-firmware-1.4.0

That gives me two manifests, and one has linux-libre and the other the
rest of the sources, but the ath9k_htc firmwares are missing.
> $ sudo find /media/gnutoo/Guix_image -type f -iname "*manifest*"
> /media/gnutoo/Guix_image/gnu/store/prap0f3zbsa3rm6mcvqn0dqzix04yd4j-profile/manifest
> /media/gnutoo/Guix_image/gnu/store/y34lb0srz4yacn3x8i1k8pkphwk0irvs-profile/manifest

And there is no mention of ath9k:
> $ sudo grep -i ath9k /media/gnutoo/Guix_image/gnu/store/*/manifest
> $

Denis.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: License compliance when redistributing images built with 'guix system'.

[Denis 'GNUtoo' Carikli]

[-- Attachment #1: Type: text/plain, Size: 4125 bytes --]

Hi again,

In the previous approach I tried to get the manifest and it failed to
get some source like the ath9k firmware.

Parsing the system definition approach:
---------------------------------------
I then tried to parse the system definition better and I could extract
it from the firmware field for instance and then I pass the resulting
package name to guix build --source=transitive:
> ;;; Copyright © 2024 Denis 'GNUtoo' Carikli
> ;;; <GNUtoo@cyberdimension.org>
> ;;; This file is free software; you can redistribute it and/or modify
> ;;; it under the terms of the GNU General Public License as
> ;;; published by the Free Software Foundation; either version 3 of
> ;;; the License, or (at your option) any later version.
> ;;;
> ;;; GNU Guix is distributed in the hope that it will be useful, but
> ;;; WITHOUT ANY WARRANTY; without even the implied warranty of
> ;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> ;;; GNU General Public License for more details.
> (use-modules (gnu bootloader))
> (use-modules (gnu system))
> (use-modules (guix packages))
> (use-modules (minimal-system))
> 
> ;; List all the OS packages but not its dependencies. This can then be
> ;; fed to guix build --sources=transitive to provide the packages
> source code.
> 
> (let ((os minimal-operating-system))
>   (map
>    (lambda (p)
>      (display (string-append (package-name p) "\n")))
>    (append
>     (operating-system-packages os)
>     (list (operating-system-kernel os))
>     (operating-system-firmware os)
>     (operating-system-locale-libcs os)
>     (list
>      (bootloader-package
>       (bootloader-configuration-bootloader
>        (operating-system-bootloader os)))))))

But then in /gnu/store of the image I have
/gnu/store/sriajy9dya89gmfaix5n7ypc3li0lmcq-libx11-1.7.3.1

And the code above doesn't list that.

Reading the derivations and filtering the result approach:
----------------------------------------------------------

So I tried again another approach and it seems to print the right paths.

I can run it with:
> guix time-machine --commit=v1.4.0 -- repl read-derivations.scm \
> $(guix gc --derivers
> /gnu/store/5rhwm44nwvc4b3ww48b8w0i6f4l0bh1c-system/)

And it produces the ath9k source, and also
/gnu/[...]-libX11-1.7.3.1.tar.xz

Here's the code:
> ;;; Copyright © 2024 Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
> ;;;
> ;;; GNU Guix is free software; you can redistribute it and/or modify it
> ;;; under the terms of the GNU General Public License as published by
> ;;; the Free Software Foundation; either version 3 of the License, or (at
> ;;; your option) any later version.
> ;;;
> ;;; GNU Guix is distributed in the hope that it will be useful, but
> ;;; WITHOUT ANY WARRANTY; without even the implied warranty of
> ;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> ;;; GNU General Public License for more details.
> (use-modules (guix derivations))
> 
> (let ((drv-from-file (read-derivation-from-file (list-ref (program-arguments) 1))))
>   (map
>    (lambda (drv-prerequisite)
>      (map
>       (lambda (drv-output)
> 	(if (derivation-output-hash (cdr drv-output))
> 	    ((lambda _
> 	       (display
> 		(string-append
> 		 (derivation-output-path (cdr drv-output))
> 		 "\n"))))))
>        (derivation-outputs (derivation-input-derivation drv-prerequisite))))
>    (derivation-prerequisites drv-from-file)))

But now I have another problem: some of the paths don't exist. For
instance the code above lists
/gnu/[...]-ath9k-htc-firmware-1.4.0-checkout which doesn't exist.

With the previous code that parsed the system definition, it produced a
different path for the system definition which existed. And running
'guix time-machine --commit=v1.4.0 -- build --system=i686-linux --source
ath9k-htc-firmware' with and without --system also produce the same
path than the code that parsed the system definition.

I'm unsure where the issue could come from. Does someone has some
hypothesis / ideas that I could look into?

Denis.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: License compliance when redistributing images built with 'guix system'.

[Denis 'GNUtoo' Carikli]

[-- Attachment #1: Type: text/plain, Size: 570 bytes --]

On Mon, 18 Nov 2024 09:41:41 +0100
Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org> wrote:
> But now I have another problem: some of the paths don't exist. For
> instance the code above lists
> /gnu/[...]-ath9k-htc-firmware-1.4.0-checkout which doesn't exist.
I've found an idea for working around that: I can get the file name and
loop over all the packages, match a package's origin-actual-file-name
against the name above, and once we have the package we can simply run
guix build --sources=transitive with it.

I'll report if it works or not.

Denis.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: License compliance when redistributing images built with 'guix system'.

[Denis 'GNUtoo' Carikli]

[-- Attachment #1: Type: text/plain, Size: 2930 bytes --]

On Mon, 18 Nov 2024 13:38:43 +0100
Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org> wrote:

> On Mon, 18 Nov 2024 09:41:41 +0100
> Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org> wrote:
> > But now I have another problem: some of the paths don't exist. For
> > instance the code above lists
> > /gnu/[...]-ath9k-htc-firmware-1.4.0-checkout which doesn't exist.
> I've found an idea for working around that: I can get the file name
> and loop over all the packages, match a package's
> origin-actual-file-name against the name above, and once we have the
> package we can simply run guix build --sources=transitive with it.
This turned out to take way too long.

I've talked again to Simon Tournier in a Guix even and he pointed to me
the missing part: guix build can build a lot of things, including
derivations and all, all I needed to do was to pass the right path to
guix build and it would figure out what it is and build it (if it can).

So at the end this gave this small source code:
;;; Copyright © 2024 Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
;;;
;;; GNU Guix is free software; you can redistribute it and/or modify it
;;; under the terms of the GNU General Public License as published by
;;; the Free Software Foundation; either version 3 of the License, or (at
;;; your option) any later version.
;;;
;;; GNU Guix is distributed in the hope that it will be useful, but
;;; WITHOUT ANY WARRANTY; without even the implied warranty of
;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;;; GNU General Public License for more details.

(use-modules (guix build utils)
             (guix derivations)
             (guix scripts build))

(let ((drv-from-file
       (read-derivation-from-file (list-ref (program-arguments) 1))))
  (map
   (lambda (drv-prerequisite)
     (map
      (lambda (drv-output)
        (if (derivation-output-hash (cdr drv-output))
            ((lambda _
               (let ((source-path (derivation-output-path (cdr drv-output))))
                 (if (not (file-exists? source-path))
                     (guix-build source-path))
                 (if (not (elf-file? source-path))
                     (display (string-append source-path "\n"))))))))
      (derivation-outputs (derivation-input-derivation drv-prerequisite))))
   (derivation-prerequisites drv-from-file)))

It can be used with something like that:
$ cp $(guix gc --derivers $(guix system image minimal-system.scm)) image.drv
$ tar cf sources.tar $(guix repl get-sources.scm image.drv)

There might still be room for testing and improvements though but we
now at least have something that can be used to easily publish complete
and corresponding source code of images in a safe and easy way.

Thanks a lot to Simon for all the help as this help has been extremely
useful in pointing me in the right direction multiple times.

Denis.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]