* Finding the store path of a package
@ 2021-03-17 17:55 Konrad Hinsen
2021-03-17 18:27 ` zimoun
0 siblings, 1 reply; 13+ messages in thread
From: Konrad Hinsen @ 2021-03-17 17:55 UTC (permalink / raw)
To: Guix Devel
Dear Guix experts,
I wonder if there is a straightforward way to find the store path
corresponding to a package, assuming that the package actually is in the
store. I don't care if it's done via the CLI or via Guile code.
Use case: Looking at the files inside a package. What I do now is "ls
/gnu/store/*<package-name>*", but that usually lists many variants of
the package, and I don't know which of them actually is the current one.
I came up with some Guile code that does the job:
(define (store-path specification)
(let*-values (((package output)
(specification->package+output specification))
((entry)
(package->manifest-entry package output))
((l-entry)
(with-store store
(run-with-store store
(lower-manifest-entry entry (%current-system))))))
(manifest-entry-item l-entry)))
but it also downloads/builds the package if it's not yet in the store,
which is not what I want. In fact, I don't care what happens then the
package is not in the store. Returning a non-existing path is fine,
as is raising an error or returning #f.
Another attempt is "guix package –-list-installed", but this works only
for packages installed in a profile. I am shifting more and more to
on-the-fly environments, meaning that many packages in my store belong
to no profile.
Cheers,
Konrad.
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Finding the store path of a package
2021-03-17 17:55 Finding the store path of a package Konrad Hinsen
@ 2021-03-17 18:27 ` zimoun
2021-03-18 8:43 ` Konrad Hinsen
0 siblings, 1 reply; 13+ messages in thread
From: zimoun @ 2021-03-17 18:27 UTC (permalink / raw)
To: Konrad Hinsen, Guix Devel
Hi Konrad,
On Wed, 17 Mar 2021 at 18:55, Konrad Hinsen <konrad.hinsen@fastmail.net> wrote:
> I wonder if there is a straightforward way to find the store path
> corresponding to a package, assuming that the package actually is in the
> store. I don't care if it's done via the CLI or via Guile code.
does “guix build <package> -n” fit your use-case?
> but it also downloads/builds the package if it's not yet in the store,
> which is not what I want. In fact, I don't care what happens then the
> package is not in the store. Returning a non-existing path is fine,
> as is raising an error or returning #f.
Well, ’package-output’ in (guix packages) is what you need, I guess.
--8<---------------cut here---------------start------------->8---
$ guix gc -D $(guix build hello)
0.1 MB will be downloaded:
/gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10
substituting /gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10...
downloading from https://ci.guix.gnu.org/nar/lzip/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10 ...
hello-2.10 51KiB 454KiB/s 00:00 [##################] 100.0%
finding garbage collector roots...
[0 MiB] deleting '/gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10'
deleting `/gnu/store/trash'
deleting unused links...
note: currently hard linking saves 20885.94 MiB
$ guix repl
GNU Guile 3.0.5
Copyright (C) 1995-2021 Free Software Foundation, Inc.
Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.
Enter `,help' for help.
scheme@(guix-user)> ,use(gnu packages base) ; hello
scheme@(guix-user)> ,use(guix packages) ; package-output
scheme@(guix-user)> ,use(guix store) ; with-store
scheme@(guix-user)> (with-store store (package-output store hello))
$1 = "/gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10"
scheme@(guix-user)> ,q
$ ls /gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10
ls: cannot access '/gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10': No such file or directory
--8<---------------cut here---------------end--------------->8---
Hope that helps,
simon
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Finding the store path of a package
2021-03-17 18:27 ` zimoun
@ 2021-03-18 8:43 ` Konrad Hinsen
2021-03-18 10:45 ` Konrad Hinsen
0 siblings, 1 reply; 13+ messages in thread
From: Konrad Hinsen @ 2021-03-18 8:43 UTC (permalink / raw)
To: zimoun, Guix Devel
Hi Simon,
> does “guix build <package> -n” fit your use-case?
Checking... yes! I hadn't even considered using "build" when building is
exactly what I do not want to happen. But yes, it works just fine.
> Well, ’package-output’ in (guix packages) is what you need, I guess.
And that works just fine as well.
Thanks,
Konrad.
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Finding the store path of a package
2021-03-18 8:43 ` Konrad Hinsen
@ 2021-03-18 10:45 ` Konrad Hinsen
2021-03-18 12:04 ` zimoun
2021-03-20 13:46 ` Ludovic Courtès
0 siblings, 2 replies; 13+ messages in thread
From: Konrad Hinsen @ 2021-03-18 10:45 UTC (permalink / raw)
To: zimoun, Guix Devel
Hi Simon,
>> does “guix build <package> -n” fit your use-case?
>
> Checking... yes! I hadn't even considered using "build" when building is
> exactly what I do not want to happen. But yes, it works just fine.
Not quite:
$ guix build -n zziplib
substitute: /gnu/store/pwcp239kjf7lnj5i4lkdzcfcxwcfyk72-bash-minimal-5.0.16/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.utf8)
24,6 MB would be downloaded:
/gnu/store/9h9153akbd6g2520mcl2d0vgbvyfi5qm-openssl-1.1.1i
/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31
/gnu/store/s3dcqzwqaakv1yx37by9chksdbkgih17-glibc-2.31-static
/gnu/store/fbn395nfpbp4d4fr6jsbmwcx6n10kg16-python-minimal-3.8.2
/gnu/store/7hizrpdsqf6q3pjgzmi51r5vbzlijkw0-python-minimal-wrapper-3.8.2
My first impression is that "guix build -n" shows the store path of the
package only if all dependencies are in the store. Otherwise, it shows
the dependencies.
>> Well, ’package-output’ in (guix packages) is what you need, I guess.
>
> And that works just fine as well.
That one is worse because it even fetches/builds the dependencies:
$guix repl
GNU Guile 3.0.5
Copyright (C) 1995-2021 Free Software Foundation, Inc.
Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.
Enter `,help' for help.
scheme@(guix-user)> (use-modules (guix packages) (guix store) (gnu packages compression))
scheme@(guix-user)> (with-store store (package-output store zziplib))
substitute: /gnu/store/pwcp239kjf7lnj5i4lkdzcfcxwcfyk72-bash-minimal-5.0.16/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.utf8)
fetching path `/gnu/store/9h9153akbd6g2520mcl2d0vgbvyfi5qm-openssl-1.1.1i'...
/gnu/store/pwcp239kjf7lnj5i4lkdzcfcxwcfyk72-bash-minimal-5.0.16/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.utf8)
Downloading https://ci.guix.gnu.org/nar/lzip/9h9153akbd6g2520mcl2d0vgbvyfi5qm-openssl-1.1.1i...
openssl-1.1.1i 2.1MiB 3.2MiB/s 00:01 [##################] 100.0%
fetching path `/gnu/store/fbn395nfpbp4d4fr6jsbmwcx6n10kg16-python-minimal-3.8.2'...
/gnu/store/pwcp239kjf7lnj5i4lkdzcfcxwcfyk72-bash-minimal-5.0.16/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.utf8)
Downloading https://ci.guix.gnu.org/nar/lzip/fbn395nfpbp4d4fr6jsbmwcx6n10kg16-python-minimal-3.8.2...
python-minimal-3.8.2 11.9MiB 2.5MiB/s 00:05 [##################] 100.0%
...
After that, I can do
$ guix build -n zziplib
/gnu/store/fx0cdzzppd8jc09sianbq6gl1h7mxx3x-zziplib-0.13.72
Cheers,
Konrad.
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Finding the store path of a package
2021-03-18 10:45 ` Konrad Hinsen
@ 2021-03-18 12:04 ` zimoun
2021-03-20 13:46 ` Ludovic Courtès
1 sibling, 0 replies; 13+ messages in thread
From: zimoun @ 2021-03-18 12:04 UTC (permalink / raw)
To: Konrad Hinsen, Guix Devel
Hi,
In addition, I am also lost with this sequence:
1. dry-run openmpi list dependencies
2. do the wrong thing with the REPL
3. garbage collect
4. redo the dry-run and now the dependencies are not listed
Sorry Konrad, ’package-output’ is definitively wrong. Worse, the items
listed in the first dry-run and then downloaded are not in the store but
not listed by dry-run neither.
--8<---------------cut here---------------start------------->8---
$ guix build -n openmpi
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
16.0 MB would be downloaded:
/gnu/store/gm6d1vr75d4247bq834pks4ln8j3bm3l-psm2-11.2.86
/gnu/store/4i9xg8dwns6ksjcb2kxkxmj9100jl4x5-ucx-1.6.1
/gnu/store/977xygcarq06p8qsxkj686cxchf6m6pg-opensm-3.3.22
/gnu/store/hbamhqhm4vq8fp27v9qgyc9rr6pf7fyw-libnl-3.5.0
/gnu/store/1sw0s2a1ndqzvngmf1jsn19nwk96m9f9-rdma-core-33.1
/gnu/store/wlhizxq9xcfz7vdf5h75xcalabad81q6-libfabric-1.11.2
/gnu/store/y88fpc6jk6a4smxqjq0s835q5mximf02-linux-pam-1.5.1
/gnu/store/agmdf3ib9sx7wqpznxxyki0pdbyhi4p5-freeipmi-1.6.7
/gnu/store/b0x9j72z490kk0pld2ab4qa146gq4x00-numactl-2.0.14
/gnu/store/jfwfsq932r209myi2ybrhy1yzh86m6a7-json-c-0.14
/gnu/store/1s9gmhycv7xhzgym0w13rnbyv4irk2d7-hwloc-2.4.1-lib
/gnu/store/vnyxs3vvzday63hk0qmlxfaq0wikc11h-munge-0.5.14
/gnu/store/4lnskfwx6s7k8d3gr3mdq1p27z784m1r-slurm-20.11.3
/gnu/store/yzs7m97m0d224qf1x8qxg1y3j5n29c3b-psm-3.3.20170428
/gnu/store/wqcbimmq44rk882hmkj99zh43jmamps2-openmpi-4.0.5-debug
/gnu/store/g07826knh72xwsdvpim5k30pqzjkcry9-openmpi-4.0.5
--8<---------------cut here---------------end--------------->8---
--8<---------------cut here---------------start------------->8---
$ guix repl
GNU Guile 3.0.5
Copyright (C) 1995-2021 Free Software Foundation, Inc.
Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.
Enter `,help' for help.
scheme@(guix-user)> ,use(gnu packages mpi)
scheme@(guix-user)> ,use(guix store)
scheme@(guix-user)> ,use(guix packages)
scheme@(guix-user)> (with-store store (package-output store openmpi))
fetching path `/gnu/store/1s9gmhycv7xhzgym0w13rnbyv4irk2d7-hwloc-2.4.1-lib'...
Downloading https://ci.guix.gnu.org/nar/lzip/1s9gmhycv7xhzgym0w13rnbyv4irk2d7-hwloc-2.4.1-lib...
hwloc-2.4.1-lib 204KiB 513KiB/s 00:00 [##################]100.0%
[..]
$1 = "/gnu/store/g07826knh72xwsdvpim5k30pqzjkcry9-openmpi-4.0.5"
scheme@(guix-user)> ,q
--8<---------------cut here---------------end--------------->8---
--8<---------------cut here---------------start------------->8---
$ guix gc
--8<---------------cut here---------------end--------------->8---
--8<---------------cut here---------------start------------->8---
$ guix build -n openmpi
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
12.5 MB would be downloaded:
/gnu/store/fbn395nfpbp4d4fr6jsbmwcx6n10kg16-python-minimal-3.8.2
/gnu/store/7hizrpdsqf6q3pjgzmi51r5vbzlijkw0-python-minimal-wrapper-3.8.2
--8<---------------cut here---------------end--------------->8---
--8<---------------cut here---------------start------------->8---
$ guix show openmpi | recsel -p dependencies | grep slurm
+ rdma-core@33.1 slurm@20.11.3 ucx@1.6.1 valgrind@3.16.1
$ guix gc -D /gnu/store/4lnskfwx6s7k8d3gr3mdq1p27z784m1r-slurm-20.11.3
finding garbage collector roots...
deleting `/gnu/store/trash'
deleting unused links...
note: currently hard linking saves 13699.88 MiB
$ guix gc --list-dead | grep slurm
finding garbage collector roots...
determining live/dead paths...
$ guix gc --list-live | grep slurm
finding garbage collector roots...
determining live/dead paths...
$ ls /gnu/store/4lnskfwx6s7k8d3gr3mdq1p27z784m1r-slurm-20.11.3
ls: cannot access '/gnu/store/4lnskfwx6s7k8d3gr3mdq1p27z784m1r-slurm-20.11.3': No such file or directory
$ guix build -n openmpi
12.5 MB would be downloaded:
/gnu/store/fbn395nfpbp4d4fr6jsbmwcx6n10kg16-python-minimal-3.8.2
/gnu/store/7hizrpdsqf6q3pjgzmi51r5vbzlijkw0-python-minimal-wrapper-3.8.2
--8<---------------cut here---------------end--------------->8---
Cheers,
simon
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Finding the store path of a package
2021-03-18 10:45 ` Konrad Hinsen
2021-03-18 12:04 ` zimoun
@ 2021-03-20 13:46 ` Ludovic Courtès
2021-03-22 7:39 ` Konrad Hinsen
1 sibling, 1 reply; 13+ messages in thread
From: Ludovic Courtès @ 2021-03-20 13:46 UTC (permalink / raw)
To: Konrad Hinsen; +Cc: Guix Devel
Hi!
Konrad Hinsen <konrad.hinsen@fastmail.net> skribis:
>>> does “guix build <package> -n” fit your use-case?
>>
>> Checking... yes! I hadn't even considered using "build" when building is
>> exactly what I do not want to happen. But yes, it works just fine.
>
> Not quite:
>
> $ guix build -n zziplib
> substitute: /gnu/store/pwcp239kjf7lnj5i4lkdzcfcxwcfyk72-bash-minimal-5.0.16/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.utf8)
> 24,6 MB would be downloaded:
> /gnu/store/9h9153akbd6g2520mcl2d0vgbvyfi5qm-openssl-1.1.1i
> /gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31
> /gnu/store/s3dcqzwqaakv1yx37by9chksdbkgih17-glibc-2.31-static
> /gnu/store/fbn395nfpbp4d4fr6jsbmwcx6n10kg16-python-minimal-3.8.2
> /gnu/store/7hizrpdsqf6q3pjgzmi51r5vbzlijkw0-python-minimal-wrapper-3.8.2
>
> My first impression is that "guix build -n" shows the store path of the
> package only if all dependencies are in the store. Otherwise, it shows
> the dependencies.
Yes. In the presence of grafts, run “guix build PKG”. That always
gives you the store file name of PKG, 100% reliable!
I regularly do things like:
ls $(guix build PKG)/bin
find $(guix build PKG) -name …
But note that ‘guix build’ returns one file name per output, so in case
of multiple-output packages, you have to pipe its output through ‘head’,
‘grep’, or similar.
If you want a variant that does that without building/downloading it,
it’s also possible, though not as easily from the command line.
Ludo’.
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Finding the store path of a package
2021-03-20 13:46 ` Ludovic Courtès
@ 2021-03-22 7:39 ` Konrad Hinsen
2021-03-22 10:03 ` zimoun
2021-03-22 16:22 ` Ludovic Courtès
0 siblings, 2 replies; 13+ messages in thread
From: Konrad Hinsen @ 2021-03-22 7:39 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: Guix Devel
Hi Ludo,
> Yes. In the presence of grafts, run “guix build PKG”. That always
> gives you the store file name of PKG, 100% reliable!
At the cost of a few hours of CPU time, in the worst case.
> I regularly do things like:
>
> ls $(guix build PKG)/bin
> find $(guix build PKG) -name …
What I am looking for is the equivalent of
ls $(guix build PKG)
that fails in whatever way for packages that are not in the store, but
guarantees (1) not adding anything to the store and (2) response times
short enough for interactive user interfaces.
> If you want a variant that does that without building/downloading it,
> it’s also possible, though not as easily from the command line.
Guile is fine, no problem. But so far, I haven't found anything even at
the Guile level that respects my two conditions.
Background: I am working on a interactive UI for running reproducible
computations via Guix:
https://github.com/khinsen/guix-gtoolkit/
I'd like to implement (1) browsing package contents ("what exactly do I
get by adding "core-utils" to my environment?") and (2) searching
packages by the files they contain ("which package do I have to add to
my environment to get the ls command?"). There will be a button for
explicitly building a package, but I don't want it to happen as a side
effect when doing operations that need to be fast.
Cheers,
Konrad.
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Finding the store path of a package
2021-03-22 7:39 ` Konrad Hinsen
@ 2021-03-22 10:03 ` zimoun
2021-03-22 13:12 ` Konrad Hinsen
2021-03-22 16:22 ` Ludovic Courtès
1 sibling, 1 reply; 13+ messages in thread
From: zimoun @ 2021-03-22 10:03 UTC (permalink / raw)
To: Konrad Hinsen, Ludovic Courtès; +Cc: Guix Devel
Hi Konrad,
On Mon, 22 Mar 2021 at 08:39, Konrad Hinsen <konrad.hinsen@fastmail.net> wrote:
> Background: I am working on a interactive UI for running reproducible
> computations via Guix:
>
> https://github.com/khinsen/guix-gtoolkit/
>
> I'd like to implement (1) browsing package contents ("what exactly do I
> get by adding "core-utils" to my environment?") and (2) searching
> packages by the files they contain ("which package do I have to add to
> my environment to get the ls command?"). There will be a button for
> explicitly building a package, but I don't want it to happen as a side
> effect when doing operations that need to be fast.
Nice!
On a side note, Ricardo did recently some stuff as UI for packages,
<http://logs.guix.gnu.org/guix-hpc/2021-03-18.log#083513>
<https://elephly.net/downies/pigx-web-envy.png>
And on another side note, I would like to have the abilities to join or
intersect graphs; be able to visualize “guix graph foo bar” using D3.js,
or the intersection or the complementary of the intersection, etc.
Something similar for packages as the Ludo’s proof of concept for
services,
<https://notabug.org/civodul/guix-explorer>
Sadly, time is not respecting what we are doing without considering
him. ;-)
Cheers,
simon
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Finding the store path of a package
2021-03-22 10:03 ` zimoun
@ 2021-03-22 13:12 ` Konrad Hinsen
0 siblings, 0 replies; 13+ messages in thread
From: Konrad Hinsen @ 2021-03-22 13:12 UTC (permalink / raw)
To: zimoun, Ludovic Courtès; +Cc: Guix Devel
Hi Simon,
> On a side note, Ricardo did recently some stuff as UI for packages,
>
> <http://logs.guix.gnu.org/guix-hpc/2021-03-18.log#083513>
> <https://elephly.net/downies/pigx-web-envy.png>
Looks good! My project has a lot of overlap, except that it is very
intentionally not based on Web technology, but on a malleable platform
(http://gtoolkit.com/) in which you can also do data analysis and other
computation. It's more like Emacs than like a Web app.
> And on another side note, I would like to have the abilities to join or
> intersect graphs; be able to visualize “guix graph foo bar” using D3.js,
> or the intersection or the complementary of the intersection, etc.
> Something similar for packages as the Ludo’s proof of concept for
> services,
>
> <https://notabug.org/civodul/guix-explorer>
Nice! That kind of visualization is not my focus for now, but the people
who build the Glamorous Toolkit platform are very much into that and
there's a lot of support code for visually exploring software systems:
https://gtoolkit.com/usecases/software-assessment/
Cheers,
Konrad.
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Finding the store path of a package
2021-03-22 7:39 ` Konrad Hinsen
2021-03-22 10:03 ` zimoun
@ 2021-03-22 16:22 ` Ludovic Courtès
2021-03-22 17:58 ` Konrad Hinsen
2021-04-19 7:08 ` Konrad Hinsen
1 sibling, 2 replies; 13+ messages in thread
From: Ludovic Courtès @ 2021-03-22 16:22 UTC (permalink / raw)
To: Konrad Hinsen; +Cc: Guix Devel
Hi Konrad,
Konrad Hinsen <konrad.hinsen@fastmail.net> skribis:
>> Yes. In the presence of grafts, run “guix build PKG”. That always
>> gives you the store file name of PKG, 100% reliable!
>
> At the cost of a few hours of CPU time, in the worst case.
>
>> I regularly do things like:
>>
>> ls $(guix build PKG)/bin
>> find $(guix build PKG) -name …
>
> What I am looking for is the equivalent of
>
> ls $(guix build PKG)
>
> that fails in whatever way for packages that are not in the store, but
> guarantees (1) not adding anything to the store and (2) response times
> short enough for interactive user interfaces.
>
>> If you want a variant that does that without building/downloading it,
>> it’s also possible, though not as easily from the command line.
>
> Guile is fine, no problem. But so far, I haven't found anything even at
> the Guile level that respects my two conditions.
Here’s an example of how to do that:
--8<---------------cut here---------------start------------->8---
scheme@(guix-user)> ,use(guix)
scheme@(guix-user)> (define s (open-connection ))
scheme@(guix-user)> ,use(gnu packages base)
scheme@(guix-user)> (package-derivation s coreutils #:graft? #f)
$1 = #<derivation /gnu/store/k0h3zahr74ky7z4hy5nklk5ar1b841nc-coreutils-8.32.drv => /gnu/store/yvsd53rkbvy9q8ak6681hai62nm6rf31-coreutils-8.32-debug /gnu/store/n8awazyldv9hbzb7pjcw76hiifmvrpvd-coreutils-8.32 7fc814f2e1e0>
scheme@(guix-user)> (derivation-outputs $1)
$2 = (("debug" . #<<derivation-output> path: "/gnu/store/yvsd53rkbvy9q8ak6681hai62nm6rf31-coreutils-8.32-debug" hash-algo: #f hash: #f recursive?: #f>) ("out" . #<<derivation-output> path: "/gnu/store/n8awazyldv9hbzb7pjcw76hiifmvrpvd-coreutils-8.32" hash-algo: #f hash: #f recursive?: #f>))
scheme@(guix-user)> (derivation->output-path $1 "out")
$3 = "/gnu/store/n8awazyldv9hbzb7pjcw76hiifmvrpvd-coreutils-8.32"
--8<---------------cut here---------------end--------------->8---
Why #:graft? #f? Because if you enable graft, you’ll potentially have
to build/download the thing, and that wouldn’t buy you anything because
the set of file names is the same in the grafted package.
Does that make sense?
> Background: I am working on a interactive UI for running reproducible
> computations via Guix:
>
> https://github.com/khinsen/guix-gtoolkit/
Nice!
> I'd like to implement (1) browsing package contents ("what exactly do I
> get by adding "core-utils" to my environment?") and (2) searching
> packages by the files they contain ("which package do I have to add to
> my environment to get the ls command?"). There will be a button for
> explicitly building a package, but I don't want it to happen as a side
> effect when doing operations that need to be fast.
For #2, there have been discussions about building a service that would
create such a database—a mapping from file names to packages. It’s not
possible to do with purely local knowledge because, by definition, you’d
have to build/download every package. I don’t think it has materialized
yet, though.
Thanks,
Ludo’.
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Finding the store path of a package
2021-03-22 16:22 ` Ludovic Courtès
@ 2021-03-22 17:58 ` Konrad Hinsen
2021-04-19 7:08 ` Konrad Hinsen
1 sibling, 0 replies; 13+ messages in thread
From: Konrad Hinsen @ 2021-03-22 17:58 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: Guix Devel
Hi Ludo,
> Here’s an example of how to do that:
Works fine, thanks!
> Why #:graft? #f? Because if you enable graft, you’ll potentially have
> to build/download the thing, and that wouldn’t buy you anything because
> the set of file names is the same in the grafted package.
OK, so that's the secret, because that's the only difference with what
I tried before.
One day I'll figure out how grafts work, but that day is not today ;-)
> For #2, there have been discussions about building a service that would
> create such a database—a mapping from file names to packages. It’s not
> possible to do with purely local knowledge because, by definition, you’d
> have to build/download every package. I don’t think it has materialized
> yet, though.
That would certainly be the best solution, but in the meantime, I'll go
ahead with what is possible today. In practice, I expect most such
queries to succeed because there are only a few packages that contain
popular commands, and those are usually in the store.
Cheers,
Konrad.
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Finding the store path of a package
2021-03-22 16:22 ` Ludovic Courtès
2021-03-22 17:58 ` Konrad Hinsen
@ 2021-04-19 7:08 ` Konrad Hinsen
1 sibling, 0 replies; 13+ messages in thread
From: Konrad Hinsen @ 2021-04-19 7:08 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: Guix Devel, zimoun
[-- Attachment #1: Type: text/plain, Size: 3012 bytes --]
Hi Ludo,
> Why #:graft? #f? Because if you enable graft, you’ll potentially have
> to build/download the thing, and that wouldn’t buy you anything because
> the set of file names is the same in the grafted package.
Four weeks later: this mostly works, but sometimes fails (by
downloading/building package) and it rarely fails completely
(error message). And I have no idea what is going on.
As an experiment, I ran the attached script via "guix repl". It requests
and displays the store paths for all non-hidden packages. Using
"guix gc" before and after, I found that it downloads/builds
more than 50000 store entries with a total size of 7.5 GB.
One small but interesting example: ABCL
##################################################
Store paths for abcl@1.8.0:
fetching path `/gnu/store/rz42ba0my9vrgbkjpkzr2drmnjk5ah50-python-3.8.2'...
Downloading https://ci.guix.gnu.org/nar/lzip/rz42ba0my9vrgbkjpkzr2drmnjk5ah50-python-3.8.2...
python-3.8.2 12.0MiB 1.8MiB/s 00:07 [##################] 100.0%
fetching path `/gnu/store/bvd09gb8ka642jzgxd2lpqlpdp160gn0-python-wrapper-3.8.2'...
Downloading https://ci.guix.gnu.org/nar/lzip/bvd09gb8ka642jzgxd2lpqlpdp160gn0-python-wrapper-3.8.2...
python-wrapper-3.8.2 347B 177KiB/s 00:00 [##################] 100.0%
fetching path `/gnu/store/h8z924ip7ialjhd3sc2id5yh0jy7cj20-python-nose-1.3.7'...
Downloading https://ci.guix.gnu.org/nar/lzip/h8z924ip7ialjhd3sc2id5yh0jy7cj20-python-nose-1.3.7...
python-nose-1.3.7 197KiB 1.4MiB/s 00:00 [##################] 100.0%
fetching path `/gnu/store/cshy0265w5ifh12v9dcmlrdd3cwb61db-mercurial-5.6.1'...
Downloading https://ci.guix.gnu.org/nar/lzip/cshy0265w5ifh12v9dcmlrdd3cwb61db-mercurial-5.6.1...
mercurial-5.6.1 3.3MiB 3.1MiB/s 00:01 [##################] 100.0%
((out . "/gnu/store/nz8xi3x7n3v2k4s4piiw7r8plvxam9qz-abcl-1.8.0"))
##################################################
Why does computing the store path for a Java package require fetching
Mercurial (and thus Python)? The only link I see is icedtea-6, whose
build procedure apparently uses Mercurial to download the OpenJDK
sources. Probably this gets inherited by icedtea-8, which is used for
building ABCL. But... why does it take Mercurial to just compute the
store path??? If my understanding of Guix hashes is correct (which I am
beginning to doubt), it shouldn't require more than the hash of the
Mercurial package.
As an example for a hard failure, see sunxi-tools@1.4.2. Asking for
its store paths downloads plenty of stuff, before failing with:
while setting up the build environment: a `armhf-linux' is required to build `/gnu/store/4blcfrrzc3yx6xjvw31mhnr1r9lpw4mj-Python-3.8.2.tar.xz.drv', but I am a `x86_64-linux'
That makes sense, of course, and I don't mind getting an error in such a
case, but I'd prefer it to fail without first downloading tons of stuff,
including the heavyweight gcc-7.5.0.
Cheers,
Konrad.
[-- Attachment #2: display-store-paths.scm --]
[-- Type: application/octet-stream, Size: 888 bytes --]
(use-modules
(gnu packages)
(guix packages)
(guix derivations)
(guix store))
(define (display-store-paths specification)
(format (current-error-port)
"Store paths for ~a:\n" specification)
(let*
((package (specification->package specification))
(drv (with-store store
(package-derivation store package #:graft? #f)))
(paths (derivation->output-paths drv))
(store-paths (map (lambda (entry)
(cons (string->symbol (car entry)) (cdr entry)))
paths)))
(format (current-error-port)
"~s\n" store-paths)))
(fold-available-packages
(lambda* (name version result
#:key outputs location
supported? deprecated?
#:allow-other-keys)
(if (and supported? (not deprecated?))
(display-store-paths (format #f "~a@~a" name version))
#f))
'())
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Finding the store path of a package
@ 2024-04-22 23:58 Nikolaos Chatzikonstantinou
0 siblings, 0 replies; 13+ messages in thread
From: Nikolaos Chatzikonstantinou @ 2024-04-22 23:58 UTC (permalink / raw)
To: konrad.hinsen, guix-devel
>Ludovic Courtès wrote:
>> Yes. In the presence of grafts, run “guix build PKG”. That always
>> gives you the store file name of PKG, 100% reliable!
>At the cost of a few hours of CPU time, in the worst case.
>> I regularly do things like:
>>
>> ls $(guix build PKG)/bin
>> find $(guix build PKG) -name …
>What I am looking for is the equivalent of
> ls $(guix build PKG)
>that fails in whatever way for packages that are not in the store, but
>guarantees (1) not adding anything to the store and (2) response times
>short enough for interactive user interfaces.
You can also try guix size $PKG.
Regards,
Nikolaos Chatzikonstantinou
^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2024-05-04 17:54 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2021-03-17 17:55 Finding the store path of a package Konrad Hinsen
2021-03-17 18:27 ` zimoun
2021-03-18 8:43 ` Konrad Hinsen
2021-03-18 10:45 ` Konrad Hinsen
2021-03-18 12:04 ` zimoun
2021-03-20 13:46 ` Ludovic Courtès
2021-03-22 7:39 ` Konrad Hinsen
2021-03-22 10:03 ` zimoun
2021-03-22 13:12 ` Konrad Hinsen
2021-03-22 16:22 ` Ludovic Courtès
2021-03-22 17:58 ` Konrad Hinsen
2021-04-19 7:08 ` Konrad Hinsen
-- strict thread matches above, loose matches on Subject: below --
2024-04-22 23:58 Nikolaos Chatzikonstantinou
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).