Re: Howto reference a custom package from a manifest

[Kaelyn <kaelyn.alexi@protonmail.com>]

Hi Tim,

------- Original Message -------
On Monday, May 22nd, 2023 at 8:20 PM, Timothy Washington <twashing@gmail.com> wrote:


> Yes that was it! I added "(native-inputs (list perl python))" to my package definition.
> $ guix build -L ~/dotfiles/ rust-rustscan # A. Now builds again!
> /gnu/store/4bldy27x1f2mzjqg5jd176nrawl98y1y-rust-rustscan-2.1.1
> 
> $ guix package -L ~/dotfiles/ -m ~/dotfiles/guix/packages/manifest.scm # B. Now also builds!
> ...
> The following derivation will be built:
> /gnu/store/xll763hpl7mvdkxd3kf8f98pygarzh41-profile.drv
> 
> ...
> 
> guix package --list-profiles # C. does NOT show the custom profile just built
> /home/twashing/.config/guix/current
> /home/twashing/.guix-profile

This is expected. When you ran the "guix package -L ~/dotfiles/ -m ~/dotfiles/guix/packages/manifest.scm" command above, that not only built the manifest but also used the manifest to create a new generation of /home/twashing/.guix-profile. If you run "guix package --list-generations" you should see a list of all of the existing (numbered) generations with the currently active generation marked (it's typically the last / latest generation, and should be in this case). Running "guix package -I" will list the packages in the current generation of your default guix profile, which should contain the packages in your manifest.

To go into more details about how profiles work: unless you specify the "-p" argument to "guix package", it will operate on your default profile when installing or removing packages (including through using manifests with "-m"), and create a new generation when the set of installed packages changes. /home/twashing/.guix-profile is a symlink to /var/guix/profiles/per-user/twashing/guix-profile, which in turn is a symlink to one of the numbered symlinks in that same /var/guix directory (i.e. if your current generation number is 14, then /var/guix/profiles/per-user/twashing/guix-profile will point to /var/guix/profiles/per-user/twashing/guix-profile-14-link). Each of those numbered symlinks points to the actual profile in /gnu/store, so using that hypothetical generation 14 as the one created by your "guix package -m" command above, /var/guix/profiles/per-user/twashing/guix-profile-14-link would be a symlink to /gnu/store/sqaz4ff2nshfizfh8ymbzllia6lsgnfv-profile.

> 
> 
> cat /gnu/store/xll763hpl7mvdkxd3kf8f98pygarzh41-profile.drv # D. DOES show the new profile in /gnu/store
> Derive([("out","/gnu/store/sqaz4ff2nshfizfh8ymbzllia6lsgnfv-profile","","")], ... ("out","/gnu/store/sqaz4ff2nshfizfh8ymbzllia6lsgnfv-profile"),("preferLocalBuild","1")])
> 
> 
> i. Using a direct "guix build" gives you a directory in "/gnu/store". And you can add that bin to your PATH.

This is fragile and will work only as long as the package exists under /gnu/store. Which is to say, this method will break as soon as "guix gc" is run unless the package happens to be "live" as a member or dependency of any existing generation of the system profile or other profiles (the profile generation symlinks like the guix-profile-14-link mentioned above are the garbage collector roots for guix, as can be seen with "guix gc --list-roots").

> ii. But for "/gnu/store/*-profile/", would you just loop over those profile directories, and run each "/gnu/store/*-profile/etc/profile"?

This is a very bad idea, as many or most of those "/gnu/store/*-profile" directories are different generations of the same profile, and sourcing them all would result in many conflicts and other troubles, including having unpredictable and likely outdated versions of commands being picked up before newer counterparts.

Cheers,
Kaelyn

> 
> 
> I'm attaching the full package definition to this email.
> 
> Thanks a lot!Tim
> 
> On Mon, 22 May 2023 at 14:18, Kaelyn <kaelyn.alexi@protonmail.com> wrote:
> 
> > Hi Tim,
> > 
> > 
> > ------- Original Message -------
> > On Sunday, May 21st, 2023 at 8:35 PM, Timothy Washington <twashing@gmail.com> wrote:
> > 
> > 
> > > Hey Simon, sure thing.
> > > I've attached "shaka.scm" here. I was able to build it separately (see "Howto supply cargo-build-system dependency to guix package definition"). That was using these commands.
> > > 
> > > guix import crate -r rustscan
> > > 
> > > guix build -L ~/dotfiles/ rust-rustscan-2
> > > 
> > > 
> > > A. I re-ran "guix build". Note that I definitely installed (and sourced) perl and python3.
> > 
> > 
> > You will need to add perl and python to the native-inputs field of your rust-rustscan-2 package for it to see those two programs. When packages are built, the building happens in an isolated environment distinct from your shell environment, so packages you install through "guix package -i" or "guix install" won't be seen in the package's build environment. https://guix.gnu.org/en/manual/devel/en/html_node/package-Reference.html#package-Reference describes the various fields including three different types of inputs, but my rule of thumb is that if the package depends on and is linking to a library then the library package is an input, and if the dependency is a program that needs to be run as part of the build (such as the rustscan package trying to run perl and python3) it should be a native-input. HTH!
> > 
> > Cheers,
> > Kaelyn
> > 
> > P.S. I've not packaged any rust code, but from what I recall rust packages that use cargo-build-system are a bit anomalous in that they have to declare the rust packages they depend on in a #:cargo-inputs argument instead of the normal inputs package field.
> > 
> > 
> > > And updated my system with "guix pull && guix package -u". But now it's failing with the below.
> > > 
> > > 
> > > guix build -L ~/dotfiles/ rust-rustscan
> > > 
> > > substitute: updating substitutes from 'https://ci.guix.gnu.org'... 0.0%guix substitute: warning: ci.guix.gnu.org: connection failed: Connection timed out
> > > substitute:
> > > substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 100.0%
> > > The following derivation will be built:
> > > /gnu/store/x695f07186dwqpw2jk48b62p2s18f5ry-rust-rustscan-2.1.1.drv
> > > ...
> > > phase `patch-usr-bin-file' succeeded after 0.0 seconds
> > > starting phase `patch-source-shebangs'
> > > patch-shebang: ./fixtures/.rustscan_scripts/test_script.pl: warning: no binary for interpreter `perl' found in $PATH
> > > patch-shebang: ./fixtures/.rustscan_scripts/test_script.py: warning: no binary for interpreter `python3' found in $PATH
> > > patch-shebang: ./fixtures/.rustscan_scripts/test_script.sh: changing `/bin/bash' to `/gnu/store/rib9g2ig1xf3kclyl076w28parmncg4k-bash-minimal-5.1.16/bin/bash'
> > > phase `patch-source-shebangs' succeeded after 0.0 seconds
> > > starting phase `configure'
> > > Unpacking rust-ansi-term...
> > > error: failed to run custom build command for `ring v0.16.20`
> > > 
> > > 
> > > Caused by:
> > > process didn't exit successfully: `/tmp/guix-build-rust-rustscan-2.1.1.drv-0/rustscan-2.1.1/target/release/build/ring-9bf05aa562ef9c86/build-script-build` (exit status: 101)
> > > --- stderr
> > > running "perl" "crypto/fipsmodule/aes/asm/aesni-x86_64.pl" "elf" "/tmp/guix-build-rust-rustscan-2.1.1.drv-0/rustscan-2.1.1/target/release/build/ring-297f46c71994a65c/out/aesni-x86_64-elf.S"
> > > thread 'main' panicked at 'failed to execute ["perl" "crypto/fipsmodule/aes/asm/aesni-x86_64.pl" "elf" "/tmp/guix-build-rust-rustscan-2.1.1.drv-0/rustscan-2.1.1/target/release/build/ring-297f46c71994a65c/out/aesni-x86_64-elf.S"]: No such file or directory (os error 2)', /tmp/guix-build-rust-rustscan-2.1.1.drv-0/rustscan-2.1.1/guix-vendor/rust-ring-0.16.20.tar.xz/build.rs:653:9
> > > note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
> > > warning: build failed, waiting for other jobs to finish...
> > > error: in phase 'build': uncaught exception:
> > > %exception #<&invoke-error program: "cargo" arguments: ("build" "--release") exit-status: 101 term-signal: #f stop-signal: #f>
> > > phase `build' failed after 12.2 seconds
> > > command "cargo" "build" "--release" failed with status 101
> > > builder for `/gnu/store/x695f07186dwqpw2jk48b62p2s18f5ry-rust-rustscan-2.1.1.drv' failed with exit code 1
> > > build of /gnu/store/x695f07186dwqpw2jk48b62p2s18f5ry-rust-rustscan-2.1.1.drv failed
> > > View build log at '/var/log/guix/drvs/x6/95f07186dwqpw2jk48b62p2s18f5ry-rust-rustscan-2.1.1.drv.gz'.
> > > guix build: error: build of `/gnu/store/x695f07186dwqpw2jk48b62p2s18f5ry-rust-rustscan-2.1.1.drv' failed
> > > 
> > > B. The idea is to include that package as part of a larger profile that I can add to my base user profile. That's using the below. rust-rustscan is defined in "guix/packages/shaka.scm".
> > > And I get an error gain, when calling guix package
> > > 
> > > guix package -L ~/dotfiles/ -m ~/dotfiles/guix/packages/manifest.scm
> > > The following packages will be installed:
> > > emacs 28.2
> > > gcc-toolchain 12.3.0
> > > git 2.40.1
> > > make 4.3
> > > rust-rustscan 2.1.1
> > > tree 2.1.0
> > > ungoogled-chromium 112.0.5615.165-1
> > > vim 9.0.1384
> > > 
> > > substitute: updating substitutes from 'https://ci.guix.gnu.org'... 0.0%guix substitute: warning: ci.guix.gnu.org: connection failed: Connection timed out
> > > substitute:
> > > The following derivations will be built:
> > > /gnu/store/v5f24ky0nw76yy24jx1ihggs6yffnvvb-profile.drv
> > > /gnu/store/x695f07186dwqpw2jk48b62p2s18f5ry-rust-rustscan-2.1.1.drv
> > > 
> > > ...
> > > cannot build derivation `/gnu/store/v5f24ky0nw76yy24jx1ihggs6yffnvvb-profile.drv': 1 dependencies couldn't be built
> > > guix package: error: build of `/gnu/store/v5f24ky0nw76yy24jx1ihggs6yffnvvb-profile.drv' failed
> > > 
> > > 
> > > Thanks
> > > Tim
> > > 
> > > On Sun, 21 May 2023 at 12:46, Simon Tournier <zimon.toutoune@gmail.com> wrote:
> > > 
> > > > Hi,
> > > > 
> > > > Please keep CC guix-devel.
> > > > 
> > > > On Sun, 21 May 2023 at 04:44, Timothy Washington <twashing@gmail.com> wrote:
> > > > 
> > > > > $ cat guix/packages/manifest.scm
> > > > > (define-module (guix packages manifest))
> > > > > (use-modules (gnu packages))
> > > > 
> > > > Well, I am not sure these two lines are required.
> > > > 
> > > > > building /gnu/store/x695f07186dwqpw2jk48b62p2s18f5ry-rust-rustscan-2.1.1.drv...
> > > > > \ 'build' phasebuilder for `/gnu/store/x695f07186dwqpw2jk48b62p2s18f5ry-rust-rustscan-2.1.1.drv' failed with exit code 1
> > > > 
> > > > It means your custom package rust-rustscan does not build.
> > > > 
> > > > Could you share your local file: ./guix/packages/shaka.scm containing
> > > > the definition of this custom package?
> > > > 
> > > > 
> > > > Cheers,
> > > > simon