bug#52684: [BUG] Multiple Packages Failing to Build

bug#52684: [BUG] Multiple Packages Failing to Build

[Christopher Rodriguez]

[-- Attachment #1.1.1: Type: text/plain, Size: 2121 bytes --]

                __________________________________________

                 [BUG] MULTIPLE PACKAGES FAILING TO BUILD

                                 rodnchr
                __________________________________________


Table of Contents
_________________

1. Environment
2. Steps to reproduce
3. Expected Result
4. Actual Result
5. Visual Proof (screenshots, videos, text)
6. Severity/Priority


1 Environment
=============

   - *Device:* Linode Virtual Server, and Lenovo Thinkpad E14.
   - *OS:* Guix System, and a workplace-modified version of Ubuntu.
   - *Commits:* guix: `2c469f04a3bec27b1f49680c638814bc06075b9a`.
   - *Substitutes:* Enabled, but unavailable for these packages.
   - *Additional Channels:* yewscion (personal channel):
     `84d82fede37110a34e2df5e1712eb3bf934936cf`.
   - *Blast Radius:* apl beets-bandcamp frotz nomad passwordsafe.


2 Steps to reproduce
====================

   1. *Update:* Run `guix pull`.
   2. *Upgrade:* Run `guix package -u apl nomad beets-bandcamp frotz
      passwordsafe`.


3 Expected Result
=================

   All packages should build and install properly on both System and
   Binary Installs.


4 Actual Result
===============

   *Failure:* Build of `apl` fails, complaining. If `--keep-going` is
   specified, the above-listed packages all fail as well, though nothing
   else does. As a result, I either have to remove all of the above
   packages from my profile (which I did, and which let me upgrade but
   prevented my use of those packages) or pin my system to an earlier
   commit (I have been using the attached `channels.scm` file, which
   still contains the commented commit line).


5 Visual Proof (screenshots, videos, text)
==========================================

   Attached are the failed build logs, as well as a log of stdout during
   the attempted build, and of the output of `guix describe`.


6 Severity/Priority
===================

   5 (Lowest Priority)


[-- Attachment #1.1.2: 2021-12-20-guix-describe.log --]
[-- Type: text/x-log, Size: 363 bytes --]

Generation 53	Dec 20 2021 11:13:43	(current)
  guix 2c469f0
    repository URL: https://git.savannah.gnu.org/git/guix.git
    branch: master
    commit: 2c469f04a3bec27b1f49680c638814bc06075b9a
  yewscion 84d82fe
    repository URL: https://git.sr.ht/~yewscion/yewscion-guix-channel
    branch: trunk
    commit: 84d82fede37110a34e2df5e1712eb3bf934936cf

[-- Attachment #1.1.3: 2021-12-20-build.log --]
[-- Type: text/x-log, Size: 4390 bytes --]

The following packages will be upgraded:
   apl            (dependencies or package changed)
   beets          (dependencies or package changed)
   beets-bandcamp (dependencies or package changed)
   frotz          (dependencies or package changed)
   nomad          (dependencies or package changed)
   passwordsafe   (dependencies or package changed)
   python-pyqt    (dependencies or package changed)

The following derivations will be built:
   /gnu/store/m2lcg448yhgvv45kwr9hv3sn7ph3v15v-profile.drv
   /gnu/store/7qsgqn0r9w8d6ilnsgwdmrvs13l6q3qj-nomad-0.2.0-alpha-199-g3e7a475.drv
   /gnu/store/8b7p5m8jaycc0bfklkh2rg75r989m323-frotz-2.44.drv
   /gnu/store/9l47ki9g4y3xd33kj2l9ilnrn702v760-passwordsafe-5.0.drv
   /gnu/store/x1mx0g2cxlkmgyz4ljkkhcdcqqvidby0-apl-1.8.drv
   /gnu/store/yf47dj34q26smhwl6046xdbnn21fwvh4-beets-bandcamp-0.1.4.drv

building /gnu/store/x1mx0g2cxlkmgyz4ljkkhcdcqqvidby0-apl-1.8.drv...
| 'build' phasebuilder for `/gnu/store/x1mx0g2cxlkmgyz4ljkkhcdcqqvidby0-apl-1.8.drv' failed with exit code 1
build of /gnu/store/x1mx0g2cxlkmgyz4ljkkhcdcqqvidby0-apl-1.8.drv failed
View build log at '/var/log/guix/drvs/x1/mx0g2cxlkmgyz4ljkkhcdcqqvidby0-apl-1.8.drv.bz2'.
building /gnu/store/yf47dj34q26smhwl6046xdbnn21fwvh4-beets-bandcamp-0.1.4.drv...
/ 'sanity-check' phasebuilder for `/gnu/store/yf47dj34q26smhwl6046xdbnn21fwvh4-beets-bandcamp-0.1.4.drv' failed with exit code 1
build of /gnu/store/yf47dj34q26smhwl6046xdbnn21fwvh4-beets-bandcamp-0.1.4.drv failed
View build log at '/var/log/guix/drvs/yf/47dj34q26smhwl6046xdbnn21fwvh4-beets-bandcamp-0.1.4.drv.bz2'.
building /gnu/store/8b7p5m8jaycc0bfklkh2rg75r989m323-frotz-2.44.drv...
- 'build' phasebuilder for `/gnu/store/8b7p5m8jaycc0bfklkh2rg75r989m323-frotz-2.44.drv' failed with exit code 1
build of /gnu/store/8b7p5m8jaycc0bfklkh2rg75r989m323-frotz-2.44.drv failed
View build log at '/var/log/guix/drvs/8b/7p5m8jaycc0bfklkh2rg75r989m323-frotz-2.44.drv.bz2'.
building /gnu/store/7qsgqn0r9w8d6ilnsgwdmrvs13l6q3qj-nomad-0.2.0-alpha-199-g3e7a475.drv...
| 'configure' phasebuilder for `/gnu/store/7qsgqn0r9w8d6ilnsgwdmrvs13l6q3qj-nomad-0.2.0-alpha-199-g3e7a475.drv' failed with exit code 1
build of /gnu/store/7qsgqn0r9w8d6ilnsgwdmrvs13l6q3qj-nomad-0.2.0-alpha-199-g3e7a475.drv failed
View build log at '/var/log/guix/drvs/7q/sgqn0r9w8d6ilnsgwdmrvs13l6q3qj-nomad-0.2.0-alpha-199-g3e7a475.drv.bz2'.
building /gnu/store/9l47ki9g4y3xd33kj2l9ilnrn702v760-passwordsafe-5.0.drv...
/ 'configure' phasebuilder for `/gnu/store/9l47ki9g4y3xd33kj2l9ilnrn702v760-passwordsafe-5.0.drv' failed with exit code 1
build of /gnu/store/9l47ki9g4y3xd33kj2l9ilnrn702v760-passwordsafe-5.0.drv failed
View build log at '/var/log/guix/drvs/9l/47ki9g4y3xd33kj2l9ilnrn702v760-passwordsafe-5.0.drv.bz2'.
cannot build derivation `/gnu/store/1xwzpwv1vw9j8549f26rj7rks0akdim9-ca-certificate-bundle.drv': 5 dependencies couldn't be built
cannot build derivation `/gnu/store/7a2lqk6p3wf443f9lhpkl31fs55ldmbg-emacs-subdirs.drv': 5 dependencies couldn't be built
cannot build derivation `/gnu/store/9b9bp4x04bvzld4p2ww6i2nbsj2pr9cx-fonts-dir.drv': 5 dependencies couldn't be built
cannot build derivation `/gnu/store/na2p09zd1w022imlyyvzqmlwdzcfr2rb-gdk-pixbuf-loaders-cache-file.drv': 5 dependencies couldn't be built
cannot build derivation `/gnu/store/ykjwv9n9w4sihplar0aqvlzb91v0kvcb-ghc-package-cache.drv': 5 dependencies couldn't be built
cannot build derivation `/gnu/store/f8s6nzp4ddbc7zsxpxd5nh793bljdwla-glib-schemas.drv': 5 dependencies couldn't be built
cannot build derivation `/gnu/store/gqvviyib2zx6aplg2af2b14sqlbmaih7-gtk-icon-themes.drv': 5 dependencies couldn't be built
cannot build derivation `/gnu/store/gggg6avv7d8i4mc8qbqhzf2rhqvs53br-gtk-im-modules.drv': 5 dependencies couldn't be built
cannot build derivation `/gnu/store/qx833ihh8377k8qgxjksb2ii9pj35v3i-info-dir.drv': 5 dependencies couldn't be built
cannot build derivation `/gnu/store/w0gbiyz0ppv9nalxkpzzdn465bwfijwb-xdg-desktop-database.drv': 5 dependencies couldn't be built
cannot build derivation `/gnu/store/nwccyah2kbpfz9k6dfrp5sgfl1j8ck5l-xdg-mime-database.drv': 5 dependencies couldn't be built
cannot build derivation `/gnu/store/m2lcg448yhgvv45kwr9hv3sn7ph3v15v-profile.drv': 16 dependencies couldn't be built
guix package: error: build of `/gnu/store/m2lcg448yhgvv45kwr9hv3sn7ph3v15v-profile.drv' failed


[-- Attachment #1.1.4: 47ki9g4y3xd33kj2l9ilnrn702v760-passwordsafe-5.0.drv.bz2 --]
[-- Type: application/x-bzip, Size: 9311 bytes --]

[-- Attachment #1.1.5: sgqn0r9w8d6ilnsgwdmrvs13l6q3qj-nomad-0.2.0-alpha-199-g3e7a475.drv.bz2 --]
[-- Type: application/x-bzip, Size: 13464 bytes --]

[-- Attachment #1.1.6: 7p5m8jaycc0bfklkh2rg75r989m323-frotz-2.44.drv.bz2 --]
[-- Type: application/x-bzip, Size: 4711 bytes --]

[-- Attachment #1.1.7: 47dj34q26smhwl6046xdbnn21fwvh4-beets-bandcamp-0.1.4.drv.bz2 --]
[-- Type: application/x-bzip, Size: 3648 bytes --]

[-- Attachment #1.1.8: mx0g2cxlkmgyz4ljkkhcdcqqvidby0-apl-1.8.drv.bz2 --]
[-- Type: application/x-bzip, Size: 15020 bytes --]

[-- Attachment #1.1.9: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 4045 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

bug#52684: [BUG] Multiple Packages Failing to Build

[zimoun]

[-- Attachment #1: Type: text/plain, Size: 4228 bytes --]

Hi,

Thanks for your report.  It is collateral issue of recent core-updates
merge which major updates.  Almost was fine, except sparse packages
which seem broken.

The commit merging core-updates is 6dffced09e (where the one you mention
2c469f0 is a direct descendant), so the 2 parents are: b603554ed0 (old
master) and e3196755e6 (major updates).  Let compare availability:

--8<---------------cut here---------------start------------->8---
$ guix time-machine --commit=b603554ed0 -- weather apl beets-bandcamp frotz nomad passwordsafe
calcul de 5 dérivations de paquets pour x86_64-linux…
recherche de 5 éléments du dépôt sur https://ci.guix.gnu.org...
https://ci.guix.gnu.org
  100.0 % des substituts sont disponibles (5 sur 5)
  au moins 8,7 Mo de fichiers nar (compressés)
  11,0 Mo sur le disque (décompressé)
  0,161 secondes par requête (0,3 secondes en tout)
  6,2 requêtes par seconde

  au moins 1 000 constructions dans la queue
      aarch64-linux : 836 (83.6 %)
      i686-linux : 56 (5.6 %)
      x86_64-linux : 75 (7.5 %)
      powerpc64le-linux : 33 (3.3 %)
  vitesse de construction : .00 constructions par l'heure
      i686-linux : 0.00 constructions par heure
      x86_64-linux : 0.00 constructions par heure
recherche de 5 éléments du dépôt sur https://bordeaux.guix.gnu.org...
https://bordeaux.guix.gnu.org
  100.0 % des substituts sont disponibles (5 sur 5)
  2,5 Mo de fichiers nar (compressés)
  11,0 Mo sur le disque (décompressé)
  0,080 secondes par requête (0,3 secondes en tout)
  12,6 requêtes par seconde
  (informations sur l’intégration continue indisponibles)
--8<---------------cut here---------------end--------------->8---

and

--8<---------------cut here---------------start------------->8---
$ guix time-machine --commit=e3196755e6 -- weather apl beets-bandcamp frotz nomad passwordsafe
calcul de 5 dérivations de paquets pour x86_64-linux…
recherche de 5 éléments du dépôt sur https://ci.guix.gnu.org...
https://ci.guix.gnu.org
  0.0 % des substituts sont disponibles (0 sur 5)
  taille des substituts inconnue
  0,0 Mo sur le disque (décompressé)
  0,104 secondes par requête (0,5 secondes en tout)
  9,6 requêtes par seconde

  0.0 % (0 sur 5) des éléments manquants sont dans la queue
  au moins 1 000 constructions dans la queue
      aarch64-linux : 836 (83.6 %)
      i686-linux : 56 (5.6 %)
      x86_64-linux : 75 (7.5 %)
      powerpc64le-linux : 33 (3.3 %)
  vitesse de construction : .00 constructions par l'heure
      i686-linux : 0.00 constructions par heure
      x86_64-linux : 0.00 constructions par heure
recherche de 5 éléments du dépôt sur https://bordeaux.guix.gnu.org...
https://bordeaux.guix.gnu.org
  0.0 % des substituts sont disponibles (0 sur 5)
  taille des substituts inconnue
  0,0 Mo sur le disque (décompressé)
  0,057 secondes par requête (0,3 secondes en tout)
  17,4 requêtes par seconde
  (informations sur l’intégration continue indisponibles)
--8<---------------cut here---------------end--------------->8---

where e3196755e6 is the core-updates branch using GCC 10 as default and
many other things.


On Mon, 20 Dec 2021 at 14:17, Christopher Rodriguez <yewscion@gmail.com> wrote:

> building /gnu/store/x1mx0g2cxlkmgyz4ljkkhcdcqqvidby0-apl-1.8.drv...
> | 'build' phasebuilder for `/gnu/store/x1mx0g2cxlkmgyz4ljkkhcdcqqvidby0-apl-1.8.drv' failed with exit code 1
> build of /gnu/store/x1mx0g2cxlkmgyz4ljkkhcdcqqvidby0-apl-1.8.drv failed
> View build log at '/var/log/guix/drvs/x1/mx0g2cxlkmgyz4ljkkhcdcqqvidby0-apl-1.8.drv.bz2'.

--8<---------------cut here---------------start------------->8---
Shape.hh: In member function ‘Shape Shape::insert_axis(Axis, ShapeItem) const’:
Shape.hh:68:46: error: ‘*(const ShapeItem*)((char*)& ret + <unknown> *8 +8)’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
   68 |      loop(r, MAX_RANK)   rho[r] = other.rho[r];
      |                                   ~~~~~~~~~~~^
cc1plus: all warnings being treated as errors
--8<---------------cut here---------------end--------------->8---

Therefore, this patch fixes apl.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: fix-apl --]
[-- Type: text/x-diff, Size: 559 bytes --]

diff --git a/gnu/packages/apl.scm b/gnu/packages/apl.scm
index badec04333..a1b923053c 100644
--- a/gnu/packages/apl.scm
+++ b/gnu/packages/apl.scm
@@ -49,7 +49,8 @@ (define-public apl
        ("sqlite" ,sqlite)
        ("readline" ,readline)))
     (arguments
-     `(#:configure-flags (list (string-append
+     `(#:configure-flags (list "CXX_WERROR=no"
+                               (string-append
                                 "--with-sqlite3="
                                 (assoc-ref %build-inputs "sqlite")))))
     (synopsis "APL interpreter")

[-- Attachment #3: Type: text/plain, Size: 241 bytes --]



The other beets-bandcamp, it is because the new ’sanity-check’ phase for
Python package, so it is easy to fix.  Do you want to give a try?


For frotz, nomad and passwordsafe, they require more investigations.

Cheers,
simon

bug#52684: [BUG] Multiple Packages Failing to Build

[Christopher Rodriguez]

[-- Attachment #1.1.1: Type: text/plain, Size: 859 bytes --]

Hi Simon, thanks for taking the time to help me with this!

I am currently trying to set up a development environment for myself, 
but I've run into some issues (which I've submitted as #52708) in the 
`make check` process. I'm editing the package definition for 
`beets-bandcamp` nonetheless, but would rather make sure there is 
nothing wrong with my local setup before submitting a patch.

That said, adding `(delete 'sanity-check)` for both beets and 
beets-bandcamp does indeed make them build correctly. Is that the 
appropriate fix here, or should I be trying to make them build correctly 
with the sanity check? I'm not super-familiar with that part of python 
development.

If it's just a matter of deleting those phases, I would then be ready to 
submit a patch for that, so long as my environment issues will not cause 
a problem.

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 4045 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

bug#52684: [BUG] Multiple Packages Failing to Build

[Maxime Devos]

Christopher Rodriguez schreef op di 21-12-2021 om 12:52 [-0500]:
> That said, adding `(delete 'sanity-check)` for both beets and 
> beets-bandcamp does indeed make them build correctly. Is that the 
> appropriate fix here, or should I be trying to make them build
> correctly 
> with the sanity check? I'm not super-familiar with that part of
> python 
> development.

I don't think we should simply delete the sanity-check phase, for the
same reason that we don't set #:tests? #false when a test fails --
presumably, the sanity-check phase failing (or the failing test)
indicates a real, though possibly subtle and obscure problem.

If I run "guix build beets-bandcamp", I get:

starting phase `sanity-check'
validating 'beets-bandcamp'
/gnu/store/a8zvmrbbvs8fc3gaq9vd4b31qjpkzi5i-beets-bandcamp-
0.1.4/lib/python3.9/site-packages
...checking requirements: ERROR: beets-bandcamp==0.1.4 The 'jellyfish'
distribution was not found and is required by beets

so I presume 'python-jellyfish' needs to be added to the propagated-
inputs.  Or the 'requirements' of beets-bandcamp need can be modified
to remove jellyfish.  I don't know if beets-bandcamp actually uses
jellyfish, if it is optional and only required by parts of beets-
bandcamp, ...

Greetings,
Maxime

bug#52684: [BUG] Multiple Packages Failing to Build

[Christopher Rodriguez]

[-- Attachment #1.1.1: Type: text/plain, Size: 1656 bytes --]

Copy that, Maxime. That makes sense.

I'll work towards getting it to build without deleting that phase, then. 
Thanks for the feedback!

On Tue, Dec 21, 2021, 13:02 Maxime Devos <maximedevos@telenet.be 
<mailto:maximedevos@telenet.be>> wrote:

    Christopher Rodriguez schreef op di 21-12-2021 om 12:52 [-0500]:
     > That said, adding `(delete 'sanity-check)` for both beets and
     > beets-bandcamp does indeed make them build correctly. Is that the
     > appropriate fix here, or should I be trying to make them build
     > correctly
     > with the sanity check? I'm not super-familiar with that part of
     > python
     > development.

    I don't think we should simply delete the sanity-check phase, for the
    same reason that we don't set #:tests? #false when a test fails --
    presumably, the sanity-check phase failing (or the failing test)
    indicates a real, though possibly subtle and obscure problem.

    If I run "guix build beets-bandcamp", I get:

    starting phase `sanity-check'
    validating 'beets-bandcamp'
    /gnu/store/a8zvmrbbvs8fc3gaq9vd4b31qjpkzi5i-beets-bandcamp-
    0.1.4/lib/python3.9/site-packages
    ...checking requirements: ERROR: beets-bandcamp==0.1.4 The 'jellyfish'
    distribution was not found and is required by beets

    so I presume 'python-jellyfish' needs to be added to the propagated-
    inputs.  Or the 'requirements' of beets-bandcamp need can be modified
    to remove jellyfish.  I don't know if beets-bandcamp actually uses
    jellyfish, if it is optional and only required by parts of beets-
    bandcamp, ...

    Greetings,
    Maxime


[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 4045 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

bug#52684: [BUG] Multiple Packages Failing to Build

[Christopher Rodriguez]

[-- Attachment #1.1.1: Type: text/plain, Size: 357 bytes --]

I believe the issue was indeed beets-bandcamp missing needed 
beets-related inputs. I started by copying all of the inputs from beets, 
and then removed each one to see if it was required for beets-bandcamp 
to build successfully. The attached patch works on my cloned repo; Is 
there somewhere else I need to send this to get it reviewed and applied?

[-- Attachment #1.1.2: 0001-Added-needed-inputs-to-beets-bandcamp.patch --]
[-- Type: text/x-patch, Size: 1217 bytes --]

From fd9a031cc7d76c14c2357a142a00f759c2d3d1ca Mon Sep 17 00:00:00 2001
From: Christopher Rodriguez <yewscion@gmail.com>
Date: Tue, 21 Dec 2021 14:25:51 -0500
Subject: [PATCH] Added needed inputs to beets-bandcamp

---
 gnu/packages/music.scm | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/music.scm b/gnu/packages/music.scm
index ba0658470c..50bd77a4a2 100644
--- a/gnu/packages/music.scm
+++ b/gnu/packages/music.scm
@@ -3873,8 +3873,17 @@ (define-public beets-bandcamp
     (build-system python-build-system)
     (arguments '(#:tests? #f))          ; there are no tests
     (propagated-inputs
-     (list beets python-isodate python-beautifulsoup4 python-requests
-           python-six))
+     (list beets
+           python-beautifulsoup4
+           python-confuse
+           python-isodate
+           python-jellyfish
+           python-mediafile
+           python-munkres
+           python-musicbrainzngs
+           python-pyyaml
+           python-six
+           python-unidecode))
     (home-page "https://github.com/unrblt/beets-bandcamp")
     (synopsis "Bandcamp plugin for beets")
     (description
-- 
2.34.0


[-- Attachment #1.1.3: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 4045 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

bug#52684: [BUG] Multiple Packages Failing to Build

[Maxime Devos]

Hi,

Christopher Rodriguez schreef op di 21-12-2021 om 14:47 [-0500]:
> I believe the issue was indeed beets-bandcamp missing needed 
> beets-related inputs. I started by copying all of the inputs from beets, 
> and then removed each one to see if it was required for beets-bandcamp 
> to build successfully. [...]

Seems like the issue is that beets-bandcamp depends on beets,
and beets depends on a number of python packages in inputs not in
propagated-inputs.  So beets-bandcamp can't find indirect dependencies
IIUC.

Moving things back to propagated-inputs is not ideal because beets
is a ‘user-facing’ package with a command, not ‘merely’ a library
and propagation can cause problems (conflicts, slower profile building
...).

beets-bandcamp is also user-facing, being a plug-in, so it should also
preferably not have propagated-inputs.

There is no ideal solution here, but I think it would be acceptable
to do something like

  (inputs
    ;; Avoid propagation, here and in beets, because this package is
    ;; ‘user-facing’ and not ‘merely’ a library, see
    ;; <https://issues.guix.gnu.org/52684#6>.
    (modify-inputs (package-inputs beets)
      (append beets python-six python-requests
              python-beautifulsoup4 python-isodate)))

An additional problem is that 'beets' does not have GUIX_PYTHONPATH
in its search paths. As such, if you create a minimal environment with
only beets and beets-bandcamp and without python, GUIX_PYTHONPATH won't
contain beets-bandcamp and hence beets cannot find beets-bandcamp
(untested).

Even better would be to let 'beets' have its own search path (e.g.
GUIX_BEETSPATH) independent of GUIX_PYTHONPATH. This is implementable
by wrapping 'beets' to set GUIX_PYTHPONPATH to "dependencies of
beets:$GUIX_BEETSPATH". I don't know if wrap-program supports that
though.

Greetings,
Maxime.

bug#52684: [BUG] Multiple Packages Failing to Build

[Christopher Rodriguez]

[-- Attachment #1.1.1: Type: text/plain, Size: 2173 bytes --]

Ah, I see. That makes sense.

However, I don't think we need to necessarily use all of 'beets' inputs 
as inputs for 'beets-bandcamp', because it will build fine with just the 
inputs listed. I know it isn't DRY, but it seems like the most efficient 
way to define the package might be to simply define the packages it is 
expecting to see, and only those packages: That way, should someone 
install 'beets' and then 'beets-bandcamp' at a later time, they don't 
need to download unused inputs (like, for instance, 'python-rarfile').

That said, I suppose at least 'beets' needs to be a propagated-input for 
'beets-bandcamp', because IIUC the main difference between the 
propagated-inputs and inputs is that inputs are used only at build time 
(like 'BuildRequires' in RPM), whereas propagated-inputs are pulled in 
as installed dependencies (like 'Requires' in RPM). 'beets' would need 
to be a propagated-input because 'beets-bandcamp' is a plugin for 
'beets', and requires 'beets' to function as expected. Is that correct?

If so, I am unsure why the other originally propagated-inputs were 
listed as such when they weren't needed for beets to function. I just 
built beets-bandcamp with everything listed as a propagated-input in my 
patch moved to an input, and it built fine. Is there a way I could 
install that built version to test it, to ensure none of the inputs need 
to be propagated-inputs (aside from 'beets')?

Please let me know if I'm way off base here; I'm very new to packaging 
in GNU/Guix! (And thank You for the help while I learn!)

As for the GUIX_PYTHONPATH and GUIX_BEETSPATH idea, I would love to 
implement something like that here, but I am running against my 
inexperience here, and was unable to find useful docs on defining PATHs 
or 'wrap-program' (I haven't looked exhaustively yet, but only have so 
much time in the day to do so, unfortunately).

Could You point me to some resources to explain the mechanisms involved 
in defining PATHs, or on the 'wrap-program' function? I am more than 
willing to learn.

Sorry if I'm asking a lot of questions; I'm excited to be a part of this 
project!

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 4045 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

bug#52684: [BUG] Multiple Packages Failing to Build

[Maxime Devos]

Christopher Rodriguez schreef op di 21-12-2021 om 16:38 [-0500]:
> Ah, I see. That makes sense.
> 
> However, I don't think we need to necessarily use all of 'beets' inputs 
> as inputs for 'beets-bandcamp', because it will build fine with just the 
> inputs listed. I know it isn't DRY, but it seems like the most efficient 
> way to define the package might be to simply define the packages it is 
> expecting to see, and only those packages:

I don't have a strong opinion here.

>  That way, should someone 
> install 'beets' and then 'beets-bandcamp' at a later time, they don't 
> need to download unused inputs (like, for instance, 'python-rarfile').

This doesn't hold: if someone installs beets and beets-bandcamp, then
they need to download python-rarfile in any case, because beets has
python-rarfile as a reference (see later).

> That said, I suppose at least 'beets' needs to be a propagated-input for 
> 'beets-bandcamp', because IIUC the main difference between the 
> propagated-inputs and inputs is that inputs are used only at build time 
> (like 'BuildRequires' in RPM), whereas propagated-inputs are pulled in 
> as installed dependencies (like 'Requires' in RPM). 'beets' would need 
> to be a propagated-input because 'beets-bandcamp' is a plugin for 
> 'beets', and requires 'beets' to function as expected. Is that correct?

This is vaguely correct in some ways but also incorrect in other ways.

At guix core, the only runtime dependency mechanism is references,
and there's no concept of ‘build-time only’.

Basically, if the store item /gnu/store/xxxx has a file that contains
the string /gnu/store/yyyy, then /gnu/store/xxxx is said to have a
reference to /gnu/store/yyyy. Also, whenever a store item
/gnu/store/xxxx is substituted, its reference /gnu/store/yyyy is
substituted as well.  And having /gnu/store/xxxx in the store prevents
/gnu/store/yyyy from being garbage collected.

This works very well for C and C++ things, where we have nice things
like RUNPATH for libraries and other binaries. But for python, guile,
etc. things, there's a snag: the ‘compiled code’ is (almost) exactly
the source code shuffled around in some directory (possibly with
bytecode compilation, but the bytecode typically doesn't have an
equivalent to RUNPATH), so Guix doesn't find a reference between
/gnu/store/xxxx and /gnu/store/yyyy.

To work around this, there's propagated-inputs: if 'xxx' has 'yyy' in
its propagated-inputs, then whenever 'xxx' is installed in a profile,
'yyy' is installed in the profile as well.

(This is very different from ‘classical’ distros!)

About letting 'beets-bandcamp' propagate 'beets': that would make some
amount of sense, but we don't let, say 'emacs-minion' propagate
'emacs', we don't let 'python-six' propagate 'python'. That is, when
the user asks to install a plugin, we currently only install the plugin
and not the application it extends.

A bug? Maybe, because a plugin is useless without the corresponding
applications. Maybe not, because propagated-inputs are inconvenient in
many ways in Guix; they are a work-around to be avoided. In any case,
this is not specific to beets, so if you'd like to change this, I
suggest starting a thread on guix-devel.

> If so, I am unsure why the other originally propagated-inputs were 
> listed as such when they weren't needed for beets to function.

(The non-optional ones) are needed for beets to function.
How did 'beets' find its dependencies then when they aren't propagated?
Because 'beets' was wrapped to set GUIX_PYTHONPATH (see 'wrap' in
python-build-system.scm), and because of references (see above).

>  I just 
> built beets-bandcamp with everything listed as a propagated-input in my 
> patch moved to an input, and it built fine. Is there a way I could 
> install that built version to test it, to ensure none of the inputs need 
> to be propagated-inputs (aside from 'beets')?

A pure temporary environment
(guix shell --pure beets beets-bandcamp -- beet $do-stuff-with-
bandcamp) can be useful. Not sure about what '$do-stuff-with-bandcamp'
would be, I'm not familiar with beets.

> Please let me know if I'm way off base here; I'm very new to packaging 
> in GNU/Guix! (And thank You for the help while I learn!)
> 
> As for the GUIX_PYTHONPATH and GUIX_BEETSPATH idea, I would love to 
> implement something like that here, but I am running against my 
> inexperience here, and was unable to find useful docs on defining PATHs 
> or 'wrap-program' (I haven't looked exhaustively yet, but only have so 
> much time in the day to do so, unfortunately).
> 
> Could You point me to some resources to explain the mechanisms involved 
> in defining PATHs, or on the 'wrap-program' function? I am more than 
> willing to learn.

It appears that 'search-path-specification' is not documented in the
manual, but you could look at (guix search-paths) and look for examples
in packages (git grep -F search-path is useful). 'wrap-program' is
unfortunately undocumented as well.

Anyway, a separate GUIX_BEETSPATH isn't strictly necessary, though I
believe it would be a slight improvement. But now I think about a bit
more, I don't think it would be useful, because GUIX_BEETSPATH and
GUIX_PYTHONPATH would look in the same subdirectories anyway ...

(Search paths are bound to the ‘consuming’ package, not the ‘providing’
package.)


Greetings,
Maxime.

bug#52684: [BUG] Multiple Packages Failing to Build

[Christopher Rodriguez]

[-- Attachment #1.1.1: Type: text/plain, Size: 3680 bytes --]

So, I took some time to do some digging this morning, and I now have a 
few results and a few more questions.

First, I tried `guix shell --pure python beets beets-bandcamp` to ensure 
that the plugin would be detected once `GUIX_PYTHONPATH` was set as You 
had mentioned. That did work, though the environment did not have 
`python-isodate`, which the plugin complained was missing. If this is 
the case, should python-isodate then be a progagated input, since it is 
apparently required at runtime? Or is there a difference w/r/t the
`--pure` environment that won't be present on an actual install?

Second, I ran `cat $(which beet)` on my currently installed `beets` 
package for the `PYTHON_PATH` that is currently being set. It has a lot 
of entries, all like the following: 
`/gnu/store/8df74df68i14lfjsny07x7cq6ffn0fs5-beets-1.5.0/lib/python3.8/site-packages:/gnu/store/nc3rprg62sigx8s9ps02wb8zbaz8qzl3-python-flask-1.1.2/lib/python3.8/site-packages`. 
I'm currently diving each of these packages to see how I might add code 
to set `beets-bandcamp` to add to this list, though I wonder if that is 
actually possible, since the plugin will usually be installed after the 
root program. Is this path pulled from a global location, or is it 
determined at install time?

Third, I did try pulling

```scheme
(native-search-paths
   (list (guix-pythonpath-search-path)))
```

into the `beets` definition, but that brought a *lot* of other 
dependencies in. Same with using

```scheme
(native-inputs 
 

   `(("sitecustomize.py" ,(local-file (search-auxiliary-file 
 

        "python/sitecustomize.py")))))
```

Sorry if I am missing something obvious, here. I will keep looking, just 
wanted to log what I've tried so far.

Once I figure out how search paths actually work inside of `guix`, I'm 
sure the solution will be fairly simple. But I haven't quite gotten 
there yet.

It seems like the solution has to do with `wrap-package`, but looking at 
the `beets` package definition, I only see the following lines related 
to wrapping the binary:

```scheme
          ;; Wrap the executable, so it can find python-gi (aka 
 

          ;; pygobject) and gstreamer plugins. 
 

          (add-after 'wrap 'wrap-typelib
            (lambda* (#:key outputs #:allow-other-keys)
              (let ((prog (string-append (assoc-ref outputs "out")
                                         "/bin/beet"))
                    (plugins (getenv "GST_PLUGIN_SYSTEM_PATH"))
                    (types (getenv "GI_TYPELIB_PATH")))
                (wrap-program prog
                  `("GST_PLUGIN_SYSTEM_PATH" ":" prefix (,plugins))
                  `("GI_TYPELIB_PATH" ":" prefix (,types)))
                #t))))))
```
It doesn't seem like PYTHONPATH is being interacted with here. Is that 
something that might be defined elsewhere? I'm tempted to merge in the 
following, which I pulled from another package (solfege), to explicitly 
set PYTHONPATH… But I don't want to change something if it is set elsewhere.


```scheme
          (add-after 'install 'wrap-program
            (lambda* (#:key inputs outputs #:allow-other-keys)
              ;; Make sure 'solfege' runs with the correct PYTHONPATH. 
 

              (let* ((out (assoc-ref outputs "out"))
                     (path (getenv "GUIX_PYTHONPATH")))
                (wrap-program (string-append out "/bin/solfege")
                  `("GUIX_PYTHONPATH" ":" prefix (,path))))
              #t)))))
```

Any input or clarification will be appreciated! I'm learning a lot as I 
(slowly) work through this! Thank You for Your time!

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 4045 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

bug#52684: [BUG] Multiple Packages Failing to Build

[Maxime Devos]

Hi,

Christopher Rodriguez schreef op wo 22-12-2021 om 12:07 [-0500]:
> So, I took some time to do some digging this morning, and I now have a 
> few results and a few more questions.
> 
> First, I tried `guix shell --pure python beets beets-bandcamp` to ensure 
> that the plugin would be detected once `GUIX_PYTHONPATH` was set as You 
> had mentioned. That did work, though the environment did not have 
> `python-isodate`, which the plugin complained was missing. If this is 
> the case, should python-isodate then be a progagated input, since it is 
> apparently required at runtime?
> 

Yes, but not for that reason. If it's required at runtime, it needs to
be in 'inputs' or 'propagated-inputs'. Ideally, 'inputs' would be
sufficient, but python doesn't have RUNPATH so we have to do with
propagation.

It's probably possible to modify the plugins to adjust the load path
when the plugin is loaded, avoiding the need of propagation, though
that might be fragile and complicated. (I don't have any examples of
packages currently doing this.)

>  Or is there a difference w/r/t the
> `--pure` environment that won't be present on an actual install?

The --pure just makes sure you don't reuse old environment variables,
which is useful to make sure there aren't any missing propagated-inputs
(or anything that needs to be substitute*-ed, but no substitution
appears to be required in beets).

It is largely equivalent to removing all packages in your user profile
and system profile (including things like 'bash' and 'coreutils'!) and
then installing python, beets and beets-bandcamp, except that
"guix shell --pure" is way more convenient.

> Second, I ran `cat $(which beet)` on my currently installed `beets` 
> package for the `PYTHON_PATH` that is currently being set. It has a lot 
> of entries, all like the following: 
> `/gnu/store/8df74df68i14lfjsny07x7cq6ffn0fs5-beets-1.5.0/lib/python3.8/site-packages:/gnu/store/nc3rprg62sigx8s9ps02wb8zbaz8qzl3-python-flask-1.1.2/lib/python3.8/site-packages`. 

> I'm currently diving each of these packages to see how I might add code 
> to set `beets-bandcamp` to add to this list, though I wonder if that is 
> actually possible, since the plugin will usually be installed after the 
> root program. Is this path pulled from a global location, or is it 
> determined at install time?

You cannot retroactively modify store items.
I.e., the build code of 'beets-bandcamp' cannot modify the 'beet'
binary. If that were possible, it would open a can of reproducibility
and security worms (think e.g. of multi-user systems -- they are rare,
but they do exist).

The path in a wrapper of a package P is determined when the package P
is built.

> Third, I did try pulling
> 
> ```scheme
> (native-search-paths
>    (list (guix-pythonpath-search-path)))
> ```
> 
> into the `beets` definition, but that brought a *lot* of other 
> dependencies in. Same with using

Not sure what you mean here. A lot more dependencies in GUIX_PYTHONPATH
in the environment variables of the profile, or in the wrapper, or ...?

> ```scheme
> (native-inputs 
>    `(("sitecustomize.py" ,(local-file (search-auxiliary-file 
>         "python/sitecustomize.py")))))
> ```

I don't know how sitecustomize.py works.

> Sorry if I am missing something obvious, here. I will keep looking, just 
> wanted to log what I've tried so far.
> 
> Once I figure out how search paths actually work inside of `guix`, I'm 
> sure the solution will be fairly simple. But I haven't quite gotten 
> there yet.
> 
> It seems like the solution has to do with `wrap-package`, but looking at 
> the `beets` package definition, I only see the following lines related 
> to wrapping the binary:
> 
> ```scheme
>           ;; Wrap the executable, so it can find python-gi (aka 
>           ;; pygobject) and gstreamer plugins. 
>           (add-after 'wrap 'wrap-typelib
>             (lambda* (#:key outputs #:allow-other-keys)
>               (let ((prog (string-append (assoc-ref outputs "out")
>                                          "/bin/beet"))
>                     (plugins (getenv "GST_PLUGIN_SYSTEM_PATH"))
>                     (types (getenv "GI_TYPELIB_PATH")))
>                 (wrap-program prog
>                   `("GST_PLUGIN_SYSTEM_PATH" ":" prefix (,plugins))
>                   `("GI_TYPELIB_PATH" ":" prefix (,types)))
>                 #t))))))

Look in python-build-system.scm, the build system also does some
wrapping (you can wrap a program multiple times!).

> ```
> It doesn't seem like PYTHONPATH is being interacted with here. Is that 
> something that might be defined elsewhere?
> 

python-build-system sets GUIX_PYTHONPATH. I don't know the difference
between PYTHONPATH and GUIX_PYTHONPATH. Some compatibility mechanism
between foreign distros and Guix I presume.

>  I'm tempted to merge in the 
> following, which I pulled from another package (solfege), to explicitly 
> set PYTHONPATH… But I don't want to change something if it is set elsewhere.
> 
> 
> ```scheme
>           (add-after 'install 'wrap-program
>             (lambda* (#:key inputs outputs #:allow-other-keys)
>               ;; Make sure 'solfege' runs with the correct PYTHONPATH. 
>               (let* ((out (assoc-ref outputs "out"))
>                      (path (getenv "GUIX_PYTHONPATH")))
>                 (wrap-program (string-append out "/bin/solfege")
>                   `("GUIX_PYTHONPATH" ":" prefix (,path))))
>               #t)))))
> ```

See python-build-system and some replies above ;-.
> Any input or clarification will be appreciated! I'm learning a lot as I 
> (slowly) work through this! Thank You for Your time!

Greetings,
Maxime.

bug#52684: [BUG] Multiple Packages Failing to Build

[Christopher Rodriguez]

[-- Attachment #1.1.1: Type: text/plain, Size: 2522 bytes --]

I've been digging through the source, and it seems as though 
`python-build-system` does not actually set the $GUIX_PYTHONPATH 
variable in the environment. That variable seems to only be set by 
`python-2.7` and its derivatives, including `python3.9`, during a step 
after the install process.

I don't know if this is intended. If it is, that would mean that 
(currently) there is no easy way to set the $GUIX_PYTHONPATH variable 
outside of installing `python` in a profile. That would certainly be the 
most expedient way forward (and how it would be done on traditional 
distros).

Right now, beets is still calling the version of python in `/gnu/store`, 
but since it isn't installed alongside `beets` as a propagated-input the 
post-install step which sets that variable (which is called 
`install-sitecustomize.py` and relies of the package version of the 
python package being installed) is never run, and therefore the 
environment variable remains unset in the profile.

Both `guix install python` and `export 
GUIX_PYTHONPATH=/gnu/..../site-packages` work to allow the environment 
variable to be set. But it might be better and more future-proof to just 
install python in the profile as a dependency, since the program itself 
actually depends on `python` anyway.

What do You think?

Here is the relevant code, from /gnu/packages/python.scm.

```scheme

;Top of file


(define* (customize-site version)
   "Generate a install-sitecustomize.py phase, using VERSION."
   `(lambda* (#:key native-inputs inputs outputs #:allow-other-keys)
      (let* ((out (assoc-ref outputs "out"))
             (site-packages (string-append
                             out "/lib/python"
                             ,(version-major+minor version)
                             "/site-packages"))
             (sitecustomize.py (assoc-ref (or native-inputs inputs)
                                          "sitecustomize.py"))
             (dest (string-append site-packages "/sitecustomize.py")))
        (mkdir-p site-packages)
        (copy-file sitecustomize.py dest)
        ;; Set the correct permissions on the installed file, else the 
byte 

        ;; compilation phase fails with a permission denied error. 
 

        (chmod dest #o644))))


---8<-----------------------------------------------------------------
; python2.7 definition
	...
	...
          (add-after 'install 'install-sitecustomize.py
            ,(customize-site version)))))

```

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 4045 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

bug#52684: [BUG] Multiple Packages Failing to Build

[Maxime Devos]

Christopher Rodriguez schreef op wo 22-12-2021 om 14:59 [-0500]:
> I've been digging through the source, and it seems as though 
> `python-build-system` does not actually set the $GUIX_PYTHONPATH 
> variable in the environment. That variable seems to only be set by 
> `python-2.7` and its derivatives, including `python3.9`, during a
> step 
> after the install process.

> [...]

> I don't know if this is intended. If it is, that would mean that 
> (currently) there is no easy way to set the $GUIX_PYTHONPATH
> variable 
> outside of installing `python` in a profile. That would certainly be
> the 
> most expedient way forward (and how it would be done on traditional 
> distros).

python-build-system doesn't GUIX_PYTHONPATH, because that's the job of
the native-search-paths of python. When a python library is being
built, GUIX_PYTHONPATH is set because the library has python among its
(implicit) inputs. The same holds for profiles: if python and package
containing a lib/pythonVERSION/site-packages are in the same profile,
then GUIX_PYTHONPATH is set in that profile.

> Both `guix install python` and `export 
> GUIX_PYTHONPATH=/gnu/..../site-packages` work to allow the
> environment 
> variable to be set. But it might be better and more future-proof to
> just 
> install python in the profile as a dependency, since the program
> itself 
> actually depends on `python` anyway.

There's a reason why we don't ‘just propagate’ like in classical
distros: what if the user installs a version of python incompatible
with the version used by beets?

To avoid such incompatibilities (and other problems), the interpreter
of binaries (and the load path) is hard-coded at built time.

Also, if this is about plugins: adding GUIX_PYTHONPATH to beets'
native-search-paths should work (the wrapper uses 'prefix', not '=').

Greetings,
Maxime.

bug#52684: [BUG] Multiple Packages Failing to Build

[Maxime Devos]

Hi,

Christopher Rodriguez schreef op wo 22-12-2021 om 14:59 [-0500]:
> Right now, beets is still calling the version of python in
> `/gnu/store`, 
> but since it isn't installed alongside `beets` as a propagated-input
> the 
> post-install step which sets that variable (which is called 
> `install-sitecustomize.py` and relies of the package version of the 
> python package being installed) is never run, and therefore the 
> environment variable remains unset in the profile.

I took a look at the procedure 'customize-site' in
gnu/packages/python.scm and it doesn't mention the environment variable
GUIX_PYTHONPATH anywhere, so I don't understand this paragraph.

Greetings,
Maxime.

bug#52684: [BUG] Multiple Packages Failing to Build

[Maxime Devos]

Maxime Devos schreef op wo 22-12-2021 om 20:50 [+0000]:
> python-build-system doesn't GUIX_PYTHONPATH, because that's the job of
> the native-search-paths of python. [...]
> 

Correction, it does set that environment variable in 'add-installed-
pythonpath', but it only adds a few things and it doesn't look at the
package inputs.

bug#52684: [BUG] Multiple Packages Failing to Build

[Christopher Rodriguez]

[-- Attachment #1.1.1: Type: text/plain, Size: 2395 bytes --]



On 12/22/21 3:50 PM, Maxime Devos wrote:

> python-build-system doesn't GUIX_PYTHONPATH, because that's the job of
> the native-search-paths of python. When a python library is being
> built, GUIX_PYTHONPATH is set because the library has python among its
> (implicit) inputs. The same holds for profiles: if python and package
> containing a lib/pythonVERSION/site-packages are in the same profile,
> then GUIX_PYTHONPATH is set in that profile.

I think this is the main issue, here. Installing 'beets' in a profile 
doesn't install python, and therefore it doesn't trigger the 
GUIX_PYTHONPATH variable. 'beets' still runs because it is referencing a 
python in the store, not in the profile, but it can't find plugins that 
aren't installed in the core package because the GUIX_PYTHONPATH 
environment variable is not being set to point to the directory in the 
profile where they are symlinked. Setting this manually to point to 
site-packages immediately solves the issue.

> There's a reason why we don't ‘just propagate’ like in classical
> distros: what if the user installs a version of python incompatible
> with the version used by beets?
> 
> To avoid such incompatibilities (and other problems), the interpreter
> of binaries (and the load path) is hard-coded at built time.
> 
> Also, if this is about plugins: adding GUIX_PYTHONPATH to beets'
> native-search-paths should work (the wrapper uses 'prefix', not '=').

This is ingenious, and one of the reasons I really love using GNU/Guix. 
Such a clever fix to allow the user to really customize their system, 
and use their computer the way they want.

The more we discuss this, the more it seems to me that there should 
indeed be an environment variable specific to beets, to point to where 
the plugins should be linked.

We don't want to bring in python and force a user to decide between 
beets and their version of python, but (due to the way the python 
plugins are loaded in 'beets') we need to add a pointer to the 
site-packages of the python beets is using to run.

Are variables that point to places in the profile allowed? Or do they 
have to only point to the store? That would solve the chicken and egg 
problem here, where we can't hardcode the directory of the plugin into 
'beets' because the plugin is installed afterwards…

--

Christopher Rodriguez

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 4045 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

bug#52684: [BUG] Multiple Packages Failing to Build

[Maxime Devos]

Hi,

> Are variables that point to places in the profile allowed? Or do they
> have to only point to the store? That would solve the chicken and egg
> problem here, where we can't hardcode the directory of the plugin
> into 'beets' because the plugin is installed afterwards…

I don't know what you mean here. A search path has no idea what a
‘profile’ is, and the profile technically is merely a symlink to a
store item in the store that consists of the union of all packages in
the profile (+ things produced by profile hooks).

Unless you mean to let "beets" look in $HOME/.guix-profile, making
~/.guix-profile special, but see <https://issues.guix.gnu.org/38498#5>
and following responses for why this suboptimal.

Greeetings,
Maxime

bug#52684: [BUG] Multiple Packages Failing to Build

[Christopher Rodriguez]

[-- Attachment #1.1.1: Type: text/plain, Size: 799 bytes --]

Alright, I've gotten it working.

After discussing on IRC with a few different people, I decided that 
making a new beets-specific variable was probably the best way forward. 
I've created $GUIX_BEETSPLUGINS for this purpose, which points to 
`/gnu/store/xxx-profile/lib/python3.9/site-packages`, and appended it to 
the $GUIX_PYTHONPATH which wraps 'beets'.

I've also wrapped the whole definition in a `let` to programmatically 
set the python version, so it is easier to update should it change. 
Ideally, this would be set from the input python's version, but I don't 
know how to do that (yet).

I've also taken the liberty of moving all of the 
unnecessarily-propagated inputs to 'beets-bandcamp' and made them normal 
inputs.

Patch is attached, let me know what You think.

[-- Attachment #1.1.2: 0001-Added-GUIX_BEETSPLUGINS-variable.patch --]
[-- Type: text/x-patch, Size: 8511 bytes --]

From 5ebae013a269c0d82a6d3af9124514bbb0d7536f Mon Sep 17 00:00:00 2001
From: Christopher Rodriguez <yewscion@gmail.com>
Date: Tue, 21 Dec 2021 14:25:51 -0500
Subject: [PATCH] Added $GUIX_BEETSPLUGINS variable

---
 gnu/packages/music.scm | 177 ++++++++++++++++++++++++-----------------
 1 file changed, 102 insertions(+), 75 deletions(-)

diff --git a/gnu/packages/music.scm b/gnu/packages/music.scm
index ba0658470c..f0fa69964a 100644
--- a/gnu/packages/music.scm
+++ b/gnu/packages/music.scm
@@ -3780,82 +3780,98 @@ (define-public instantmusic
     (license license:expat))))
 
 (define-public beets
-  (package
-    (name "beets")
-    (version "1.5.0")
-    (source (origin
-              (method url-fetch)
-              (uri (pypi-uri "beets" version))
-              (sha256
-               (base32
-                "0arl4nc3y8iwa331hf6ggai19y8ns9pl03g5d6ac857wq2x7nzw8"))))
-    (build-system python-build-system)
-    (arguments
-     `(#:phases
-       (modify-phases %standard-phases
-         (add-after 'unpack 'set-HOME
-           (lambda _
-             (setenv "HOME" (string-append (getcwd) "/tmp"))
-             #t))
-         (replace 'check
-           (lambda* (#:key tests? #:allow-other-keys)
-             (when tests?
-               (invoke "pytest" "-v" "test"))))
-         ;; Wrap the executable, so it can find python-gi (aka
-         ;; pygobject) and gstreamer plugins.
-         (add-after 'wrap 'wrap-typelib
-           (lambda* (#:key outputs #:allow-other-keys)
-             (let ((prog (string-append (assoc-ref outputs "out")
-                                        "/bin/beet"))
-                   (plugins (getenv "GST_PLUGIN_SYSTEM_PATH"))
-                   (types (getenv "GI_TYPELIB_PATH")))
-               (wrap-program prog
-                 `("GST_PLUGIN_SYSTEM_PATH" ":" prefix (,plugins))
-                 `("GI_TYPELIB_PATH" ":" prefix (,types)))
-               #t))))))
-    (native-inputs
-     (list gobject-introspection
-           python-flask
-           python-mock
-           python-py7zr
-           python-pytest-6
-           python-responses))
-    (inputs
-     (list bash-minimal
-           gst-plugins-base
-           gst-plugins-good
-           gstreamer
-           python-confuse
-           python-jellyfish
-           python-mediafile
-           python-munkres
-           python-musicbrainzngs
-           python-pyyaml
-           python-six
-           python-unidecode
-           ;; Optional dependencies for plugins. Some of these are also required by tests.
-           python-beautifulsoup4 ; For lyrics.
-           python-discogs-client ; For discogs.
-           python-mpd2 ; For mpdstats.
-           python-mutagen ; For scrub.
-           python-langdetect ; For lyrics.
-           python-pillow ; For fetchart, embedart, thumbnails.
-           python-pyacoustid ; For chroma.
-           python-pygobject ; For bpd, replaygain.
-           python-pylast ; For lastgenre, lastimport.
-           python-pyxdg ; For thumbnails.
-           python-rarfile ; For import.
-           python-reflink ; For reflink.
-           python-requests
-           python-requests-oauthlib)) ; For beatport.
-    (home-page "https://beets.io")
-    (synopsis "Music organizer")
-    (description "The purpose of beets is to get your music collection
+  (let ((beets-python-version "3.9"))
+    (package
+      (name "beets")
+      (version "1.5.0")
+      (source (origin
+                (method url-fetch)
+                (uri (pypi-uri "beets" version))
+                (sha256
+                 (base32
+                  "0arl4nc3y8iwa331hf6ggai19y8ns9pl03g5d6ac857wq2x7nzw8"))))
+      (build-system python-build-system)
+      ;; Beets plugins are found by beets via PYTHONPATH; the following
+      ;; search path ensures that they are found even when Python is not
+      ;; present in the profile.
+      (native-search-paths
+       ;; XXX: Attempting to use (package-native-search-paths python) here would
+       ;; cause an error about python being an unbound variable in the
+       ;; tests. Instead, we set and use an explicit python version.
+       (list
+        (search-path-specification
+         (variable "GUIX_BEETSPLUGINPATH")
+         (separator #f)
+         (files (list (string-append "lib/python" beets-python-version "/site-packages"))))))
+      (arguments
+       `(#:phases
+         (modify-phases %standard-phases
+           (add-after 'unpack 'set-HOME
+             (lambda _
+               (setenv "HOME" (string-append (getcwd) "/tmp"))
+               #t))
+           (replace 'check
+             (lambda* (#:key tests? #:allow-other-keys)
+               (when tests?
+                 (invoke "pytest" "-v" "test"))))
+           ;; Wrap the executable, so it can find python-gi (aka pygobject) and
+           ;; gstreamer plugins.
+           (add-after 'wrap 'wrap-typelib
+             (lambda* (#:key outputs #:allow-other-keys)
+               (let* ((prog (string-append (assoc-ref outputs "out")
+                                           "/bin/beet"))
+                      (gst-plugins (getenv "GST_PLUGIN_SYSTEM_PATH"))
+                      (beets-plugins ":${GUIX_BEETSPLUGINPATH}")
+                      (types (getenv "GI_TYPELIB_PATH")))
+                 (wrap-program prog
+                   `("GST_PLUGIN_SYSTEM_PATH" ":" prefix (,gst-plugins))
+                   `("GI_TYPELIB_PATH" ":" prefix (,types))
+                   `("GUIX_PYTHONPATH" ":" prefix (,beets-plugins))))
+               #t)))))
+      (native-inputs
+       (list gobject-introspection
+             python-flask
+             python-mock
+             python-py7zr
+             python-pytest-6
+             python-responses))
+      (inputs
+       (list bash-minimal
+             gst-plugins-base
+             gst-plugins-good
+             gstreamer
+             python
+             python-confuse
+             python-jellyfish
+             python-mediafile
+             python-munkres
+             python-musicbrainzngs
+             python-pyyaml
+             python-six
+             python-unidecode
+             ;; Optional dependencies for plugins. Some of these are also required by tests.
+             python-beautifulsoup4 ; For lyrics.
+             python-discogs-client ; For discogs.
+             python-mpd2 ; For mpdstats.
+             python-mutagen ; For scrub.
+             python-langdetect ; For lyrics.
+             python-pillow ; For fetchart, embedart, thumbnails.
+             python-pyacoustid ; For chroma.
+             python-pygobject ; For bpd, replaygain.
+             python-pylast ; For lastgenre, lastimport.
+             python-pyxdg ; For thumbnails.
+             python-rarfile ; For import.
+             python-reflink ; For reflink.
+             python-requests
+             python-requests-oauthlib)) ; For beatport.
+      (home-page "https://beets.io")
+      (synopsis "Music organizer")
+      (description "The purpose of beets is to get your music collection
 right once and for all.  It catalogs your collection, automatically
 improving its metadata as it goes using the MusicBrainz database.
 Then it provides a variety of tools for manipulating and accessing
 your music.")
-    (license license:expat)))
+      (license license:expat))))
 
 (define-public beets-next
   (deprecated-package "beets-next" beets))
@@ -3871,10 +3887,21 @@ (define-public beets-bandcamp
                (base32
                 "0dwbdkrb9c0ppzm5s78h47ndpr88cw1k0z8fgfhkl706wazx2ddg"))))
     (build-system python-build-system)
-    (arguments '(#:tests? #f))          ; there are no tests
+    (arguments
+     `(#:tests? #f))
     (propagated-inputs
-     (list beets python-isodate python-beautifulsoup4 python-requests
-           python-six))
+     (list python-isodate))
+    (inputs
+     (list beets
+           python-beautifulsoup4
+           python-confuse
+           python-jellyfish
+           python-mediafile
+           python-munkres
+           python-musicbrainzngs
+           python-pyyaml
+           python-six
+           python-unidecode))
     (home-page "https://github.com/unrblt/beets-bandcamp")
     (synopsis "Bandcamp plugin for beets")
     (description
-- 
2.34.0


[-- Attachment #1.1.3: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 4045 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

bug#52684: [BUG] Multiple Packages Failing to Build

[Maxime Devos]

Hi,

Christopher Rodriguez schreef op ma 27-12-2021 om 13:18 [-0500]:
> +                   `("GST_PLUGIN_SYSTEM_PATH" ":" prefix (,gst-
> plugins))
> +                   `("GI_TYPELIB_PATH" ":" prefix (,types))
> +                   `("GUIX_PYTHONPATH" ":" prefix (,beets-
> plugins))))

I think this needs to be '=' instead of 'prefix', otherwise the GUIX_PYTHONPATH,
GI_TYPELIB_PATH and GST_PLUGIN_SYSTEM_PATH from the environment might interfere,
which is probably harmless most of the time but might be a source of potential
problems.

I'll look more at this patch later.

Greetings,
Maxime.

bug#52684: [BUG] Multiple Packages Failing to Build

[Christopher Rodriguez]

[-- Attachment #1.1.1: Type: text/plain, Size: 406 bytes --]

Hi Maxime,

Copy that, I can swap the 'prefix'es out for '='s. Would it be best to 
do that for all of them, to totally isolate the derivation from the 
environment?

No rush on reviewing, thank You for spending so much time on this with 
me. I have learned a lot about Guix packages working through this; 
Hopefully I can use that knowledge to help out more often.

--

Christopher Rodriguez

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 4045 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

bug#52684: [BUG] Multiple Packages Failing to Build

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 283 bytes --]

Hi,

Christopher Rodriguez schreef op ma 27-12-2021 om 13:18 [-0500]:
> +  (let ((beets-python-version "3.9"))
> +    (package [...]))

I don't see the point of this 'let' form, because 'beets-python-
version' appears to be used only once below.

Greetings,
Maxime.


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

bug#52684: [BUG] Multiple Packages Failing to Build

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 1249 bytes --]

Christopher Rodriguez schreef op ma 27-12-2021 om 13:18 [-0500]:
+      (native-search-paths
+       ;; XXX: Attempting to use (package-native-search-paths python)
here would
+       ;; cause an error about python being an unbound variable in the
+       ;; tests. Instead, we set and use an explicit python version.

This error doesn't have anything to do with profiles, but with import
cycles between modules and native-search-paths not being thunked.

+       (list
+        (search-path-specification
+         (variable "GUIX_BEETSPLUGINPATH")
+         (separator #f)
+         (files (list (string-append "lib/python" beets-python-version
"/site-packages"))))))

A downside of this approach, is that 'GUIX_BEETSPLUGINPATH' would
include unrelated python libraries. I don't know if this would be an
actual problem in practice. This could be resolved by installing beets
libraries to a non-standard location, say
"lib/beets-dependencies/site-packages".

The 'python-build-system' probably won't expect this, but that could be
resolved by adding things from "lib/beets-dependencies/site-packages"
to 'GUIX_PYTHONPATH' in a build phase.

Greetings,
Maxime

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

bug#52684: [BUG] Multiple Packages Failing to Build

[Christopher Rodriguez]

[-- Attachment #1.1.1: Type: text/plain, Size: 1184 bytes --]

Hey Maxime,

The let form was mostly included because, if possible, we should pull 
the value of `beets-python-version` from the version of python used for 
beets (because this implementation currently relies on a hardcoded and 
versioned library path in `lib/python3.9/site-packages`, it will require 
maintenance that could otherwise be avoided if we pulled the version 
from python directly.

However, I don't know how to do that, or if that is possible. If it's 
not something easily doable, then I will remove the let form for now, 
and see about improving it in the future.

As for the issue of other python libraries possibly polluting the beets 
install, AFAIK beets looks for a folder named `beetsplug` in the 
PYTHONPATH and only considers files beneath it. I think we can safely 
avoid isolating the beets plugins; Whether this is the route Guix wants 
to go is something I think should ideally be standardized across all 
packages if possible (maybe an email thread on guix-devel?), because the 
main channel as a whole will be more maintainable if we can come to some 
kind of agreement.

Let me know what You think.

--

Christopher Rodriguez

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 4045 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

bug#52684: [PATCH v1] Updated and fixed frotz package.

[Christopher Rodriguez]

---

Hello!

Was going through my open issues, saw this one was still open. Took a crack at
fixing frotz; It builds and works as expected now. Wanted to share.

Thanks for Your time!

 gnu/packages/games.scm | 66 +++++++++++++++++++++++++++---------------
 1 file changed, 43 insertions(+), 23 deletions(-)

diff --git a/gnu/packages/games.scm b/gnu/packages/games.scm
index 8e6ab03530..512871d780 100644
--- a/gnu/packages/games.scm
+++ b/gnu/packages/games.scm
@@ -7988,7 +7988,7 @@ (define (install src dst)
 (define-public frotz
   (package
     (name "frotz")
-    (version "2.44")
+    (version "2.54")
     (source (origin
               (method url-fetch)
               (uri (list (string-append
@@ -7999,30 +7999,50 @@ (define-public frotz
                           "frotz/frotz-" version ".tar.gz")))
               (sha256
                (base32
-                "1v735xr3blznac8fnwa27s1vhllx4jpz7kw7qdw1bsfj6kq21v3k"))))
+                "1vsfq9ryyb4nvzxpnnn40k423k9pdy8k67i8390qz5h0vmxw0fds"))))
     (build-system gnu-build-system)
     (arguments
-     `(#:tests? #f                      ; there are no tests
-       #:phases
-       (modify-phases %standard-phases
-         (delete 'configure)
-         (add-before 'build 'curses
-           (lambda _
-             (substitute* "Makefile"
-               (("lcurses") "lncurses"))
-             #t))
-         (replace 'install
-           (lambda* (#:key outputs #:allow-other-keys)
-             (let* ((out (assoc-ref outputs "out"))
-                    (bin (string-append out "/bin"))
-                    (man (string-append out "/share/man/man6")))
-               (install-file "frotz" bin)
-               (mkdir-p man)
-               (install-file "doc/frotz.6" man)
-               #t))))))
-    (inputs (list libmodplug libsamplerate libsndfile libvorbis ncurses))
-    (synopsis "Portable Z-machine interpreter (ncurses version) for text adventure games")
-    (description "Frotz is an interpreter for Infocom games and other Z-machine
+     `(#:tests? #f ;there are no tests
+       #:phases (modify-phases %standard-phases
+                  (delete 'configure)
+                  (replace 'build
+                    (lambda* _
+                      ;; Compile.
+                      (invoke "make" "frotz")))
+                  (add-before 'build 'patch-makefile
+                    (lambda* _
+                      (let ((makefiles (list "Makefile"
+                                             "src/common/Makefile"
+                                             "src/curses/Makefile"
+                                             "src/x11/Makefile"
+                                             "src/sdl/Makefile"
+                                             "src/dumb/Makefile"
+                                             "src/blorb/Makefile")))
+                        (map (lambda (x)
+                               (substitute* x
+                                 (("\\$\\(CC\\)") "gcc"))) makefiles))))
+                  (replace 'install
+                    (lambda* (#:key outputs #:allow-other-keys)
+                      (let* ((out (assoc-ref outputs "out")) (bin (string-append
+                                                                   out "/bin"))
+                             (man (string-append out "/share/man/man6")))
+                        (install-file "frotz" bin)
+                        (mkdir-p man)
+                        (install-file "doc/frotz.6" man)))))))
+    (native-inputs (list pkg-config))
+    (inputs (list ao
+                  libmodplug
+                  libsamplerate
+                  libsndfile
+                  libvorbis
+                  ncurses
+                  which
+                  perl
+                  pkg-config))
+    (synopsis
+     "Portable Z-machine interpreter (ncurses version) for text adventure games")
+    (description
+     "Frotz is an interpreter for Infocom games and other Z-machine
 games in the text adventure/interactive fiction genre.  This version of Frotz
 complies with standard 1.0 of Graham Nelson's specification.  It plays all
 Z-code games V1-V8, including V6, with sound support through libao, and uses

base-commit: 2873433c72ad6302a275579a646ba9635f036927
-- 
2.36.1