* bug#74015: 'guix shell --export-manifest' fails for some transformations
@ 2024-10-25 17:08 Simon Tournier
2024-11-10 11:38 ` Ludovic Courtès
0 siblings, 1 reply; 6+ messages in thread
From: Simon Tournier @ 2024-10-25 17:08 UTC (permalink / raw)
To: 74015
Hi,
I notice this:
--8<---------------cut here---------------start------------->8---
$ cat manifest.scm
(use-modules (guix transformations) (guix packages)
(guix download) (guix build-system r))
(define r-knitr
(specification->package "r-knitr"))
(define r-knitr-other
(package
(inherit r-knitr)
(version "1.47")
(source (origin
(method url-fetch)
(uri (cran-uri "knitr" version))
(sha256
(base32
"1kx6g8ddbigwx3bmq771gwi3din3fxjn59l801904kjaz6dq9pgs"))))))
(define r-ggplot2
(package
(inherit
((package-input-rewriting `((,r-knitr . ,r-knitr-other)))
(specification->package "r-ggplot2")))
(name "my-ggplot")))
(define transform
(options->transformation
'((with-c-toolchain . "r-minimal=gcc-toolchain@12")
(with-c-toolchain . "r=gcc-toolchain@12"))))
(packages->manifest
(append
(map specification->package
(list "python" "python-numpy"))
(map transform
(list (specification->package "r")
r-ggplot2))))
$ guix shell -m manifest.scm --export-manifest
guix shell: warning: transformation 'with-c-toolchain' had no effect on my-ggplot@3.5.1.
;; What follows is a "manifest" equivalent to the command line you gave.
;; You can store it in a file that you may then pass to any 'guix' command
;; that accepts a '--manifest' (or '-m') option.
(use-modules (guix transformations))
(define transform1
(options->transformation
'((with-c-toolchain . "r-minimal=gcc-toolchain@12")
(with-c-toolchain . "r=gcc-toolchain@12"))))
(packages->manifest
(list (specification->package "python")
(specification->package "python-numpy")
(transform1 (specification->package "r"))
(transform1 (specification->package "my-ggplot"))))
--8<---------------cut here---------------end--------------->8---
Well, this is incorrect. While I understand the approximation when
dealing with profiles built over time in the imperative way, here I am
missing why the transformation of ’r-ggplot2’ is skipped. And even the
export manifest is broken.
It’s skipped because:
--8<---------------cut here---------------start------------->8---
$ guix shell -m manifest.scm
$ grep ggplot $GUIX_ENVIRONMENT/manifest
("my-ggplot"
"/gnu/store/x0jbmvg0nbl7lyk8zd938rq2c7c9qkn4-my-ggplot-3.5.1"
--8<---------------cut here---------------end--------------->8---
Please note that if the symbol ’r-ggplot2’ is not named “my-ggplot” but
kept as ’r-ggplot2“:
(define r-ggplot2
((package-input-rewriting `((,r-knitr . ,r-knitr-other)))
(specification->package "r-ggplot2")))
then the bug is maybe worse because the exported manifest “works” but
without the transformation… which can be misleading and hard to detect.
Well, I understand it’s hard to capture this kind of transformation.
Still, it appears to me as an half-baked feature. :-)
That’s annoying in some context as “guix pack --save-provenance” [1].
Even, it defeats the idea of a self-contained reproducible binary
container. ;-)
Maybe, aside the profile ’manifest’ file, we could store all the
manifests provided by the command line. Something as:
/gnu/store/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx-profile/manifest
/gnu/store/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx-profile/manifest.orig1
/gnu/store/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx-profile/manifest.orig2
…
where manifest.orig1 and manifest.orig2 are the copy of files ’foo.scm’
and ’bar.scm’ from:
-m foo.scm -m bar.scm
This way, the option --export-manifest could be improved.
WDYT?
Cheers,
simon
1: https://hpc.guix.info/blog/2021/10/when-docker-images-become-fixed-point
^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#74015: 'guix shell --export-manifest' fails for some transformations
2024-10-25 17:08 bug#74015: 'guix shell --export-manifest' fails for some transformations Simon Tournier
@ 2024-11-10 11:38 ` Ludovic Courtès
2024-11-10 13:06 ` Simon Tournier
0 siblings, 1 reply; 6+ messages in thread
From: Ludovic Courtès @ 2024-11-10 11:38 UTC (permalink / raw)
To: Simon Tournier; +Cc: 74015
Hi,
Simon Tournier <zimon.toutoune@gmail.com> skribis:
> $ guix shell -m manifest.scm --export-manifest
‘--export-manifest’ is meant to “translate” a command line, which it can
do faithfully; there’s no way it could possibly “recreate” what
‘manifest.scm’, which may contain arbitrary Scheme code (and someone who
already has a ‘manifest.scm’ file probably doesn’t need
‘--export-manifest’).
It’s necessarily a lossy process. Some of the transformations
‘manifest.scm’ uses may be recorded as metadata in manifest entries, but
others, such as uses of ‘package-input-rewriting’ here, inevitably get
lost.
I would close it as not-a-bug; WDYT?
Ludo’.
^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#74015: 'guix shell --export-manifest' fails for some transformations
2024-11-10 11:38 ` Ludovic Courtès
@ 2024-11-10 13:06 ` Simon Tournier
2024-11-12 15:33 ` Suhail Singh
0 siblings, 1 reply; 6+ messages in thread
From: Simon Tournier @ 2024-11-10 13:06 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 74015
Hi,
On Sun, 10 Nov 2024 at 12:38, Ludovic Courtès <ludo@gnu.org> wrote:
>> $ guix shell -m manifest.scm --export-manifest
>
> ‘--export-manifest’ is meant to “translate” a command line, which it can
> do faithfully; there’s no way it could possibly “recreate” what
> ‘manifest.scm’, which may contain arbitrary Scheme code (and someone who
> already has a ‘manifest.scm’ file probably doesn’t need
> ‘--export-manifest’).
Yes, one might need. :-)
As I explained: « That’s annoying in some context as “guix pack
--save-provenance” [1]. Even, it defeats the idea of a self-contained
reproducible binary container. »
For sure, I understand it’s hard to capture this kind of transformation.
Still, an improvement appears to me possible (see below).
1: https://hpc.guix.info/blog/2021/10/when-docker-images-become-fixed-point
> It’s necessarily a lossy process.
This is where I disagree. :-) I mean, yes I agree that building
profile/manifest is somehow a lossy process because some Scheme is
potentially evaluated on the road. However, I am proposing: Aside the
profile ’manifest’ file (lossy process), we could store all the
manifests provided by the command line. Something as:
/gnu/store/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx-profile/manifest
/gnu/store/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx-profile/manifest.orig1
/gnu/store/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx-profile/manifest.orig2
…
where manifest.orig1 and manifest.orig2 are the copy of files ’foo.scm’
and ’bar.scm’ from:
-m foo.scm -m bar.scm
This way, the option --export-manifest could be improved, eventually.
In other word, I think that saving as-is the manifests costs almost
nothing and it paves the way to more robust self-contained binary packs.
Well, maybe this extra work could be only done when there is an option
’--save-provenance’ applied.
> I would close it as not-a-bug; WDYT?
I still think it’s a feature that it’s possible to improve.
Cheers,
simon
^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#74015: 'guix shell --export-manifest' fails for some transformations
2024-11-10 13:06 ` Simon Tournier
@ 2024-11-12 15:33 ` Suhail Singh
2024-11-16 7:20 ` Simon Tournier
0 siblings, 1 reply; 6+ messages in thread
From: Suhail Singh @ 2024-11-12 15:33 UTC (permalink / raw)
To: Simon Tournier; +Cc: Ludovic Courtès, 74015
Simon Tournier <zimon.toutoune@gmail.com> writes:
> However, I am proposing: Aside the profile ’manifest’ file (lossy
> process), we could store all the manifests provided by the command
> line. Something as:
>
> /gnu/store/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx-profile/manifest
> /gnu/store/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx-profile/manifest.orig1
> /gnu/store/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx-profile/manifest.orig2
> …
>
> where manifest.orig1 and manifest.orig2 are the copy of files ’foo.scm’
> and ’bar.scm’ from:
>
> -m foo.scm -m bar.scm
>
> ...
>
> Well, maybe this extra work could be only done when there is an option
> ’--save-provenance’ applied.
For --save-provenance, is your proposal that in addition to saving the
additional manifests, the commandline that was used would also be saved?
--
Suhail
^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#74015: 'guix shell --export-manifest' fails for some transformations
2024-11-12 15:33 ` Suhail Singh
@ 2024-11-16 7:20 ` Simon Tournier
2024-11-17 19:05 ` Suhail Singh
0 siblings, 1 reply; 6+ messages in thread
From: Simon Tournier @ 2024-11-16 7:20 UTC (permalink / raw)
To: Suhail Singh; +Cc: Ludovic Courtès, 74015
Hi,
On Tue, 12 Nov 2024 at 10:33, Suhail Singh <suhailsingh247@gmail.com> wrote:
> Simon Tournier <zimon.toutoune@gmail.com> writes:
>
>> However, I am proposing: Aside the profile ’manifest’ file (lossy
>> process), we could store all the manifests provided by the command
>> line. Something as:
>>
>> /gnu/store/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx-profile/manifest
>> /gnu/store/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx-profile/manifest.orig1
>> /gnu/store/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx-profile/manifest.orig2
>> …
>>
>> where manifest.orig1 and manifest.orig2 are the copy of files ’foo.scm’
>> and ’bar.scm’ from:
>>
>> -m foo.scm -m bar.scm
>>
>> ...
>>
>> Well, maybe this extra work could be only done when there is an option
>> ’--save-provenance’ applied.
>
> For --save-provenance, is your proposal that in addition to saving the
> additional manifests, the commandline that was used would also be saved?
We could save the command line too using another file. :-)
Cheers,
simon
^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#74015: 'guix shell --export-manifest' fails for some transformations
2024-11-16 7:20 ` Simon Tournier
@ 2024-11-17 19:05 ` Suhail Singh
0 siblings, 0 replies; 6+ messages in thread
From: Suhail Singh @ 2024-11-17 19:05 UTC (permalink / raw)
To: Simon Tournier; +Cc: Ludovic Courtès, Suhail Singh, 74015
Simon Tournier <zimon.toutoune@gmail.com> writes:
>> For --save-provenance, is your proposal that in addition to saving the
>> additional manifests, the commandline that was used would also be saved?
>
> We could save the command line too using another file. :-)
That would be helpful :)
Given an image generated with --save-provenance enabled, making it
easier for it to be reconstructed by another would certainly be useful.
--
Suhail
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2024-11-17 19:07 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-10-25 17:08 bug#74015: 'guix shell --export-manifest' fails for some transformations Simon Tournier
2024-11-10 11:38 ` Ludovic Courtès
2024-11-10 13:06 ` Simon Tournier
2024-11-12 15:33 ` Suhail Singh
2024-11-16 7:20 ` Simon Tournier
2024-11-17 19:05 ` Suhail Singh
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/guix.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.