unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
* Upgrading SIP.
@ 2021-02-22 11:50 Brendan Tildesley
  2021-02-22 17:50 ` Ludovic Courtès
  0 siblings, 1 reply; 3+ messages in thread
From: Brendan Tildesley @ 2021-02-22 11:50 UTC (permalink / raw)
  To: guix-devel@gnu.org


[-- Attachment #1.1: Type: text/plain, Size: 1755 bytes --]

I'm looking at upgrading python-sip, since I think Calibre may need it, and presumably it needs updating anyway. Updating the package its self was trivial, but when building packages with it I came across some issues that seem unique to Guix/Nix's directory structure. SIP 5/6 uses sip-build instead of `python configure.py ...'.

Updating python-pyqt to use sip@6 mailto:sip@6 with sip-build results in some binaries being installed to a doubled up  /gnu/store/.../gnu/store/...-python-wrapper.../bin/... path, since I used INSTALL_ROOT = %output, and set sip-build --target-dir to a relative path.

Building python-pyqtwebengine, it is unable to find the .sip files needed since it only searches one global path:

openat(AT_FDCWD, "QtCore/QtCoremod.sip", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/tmp/guix-build-python-pyqtwebengine-5.15.2.drv-0/PyQtWebEngine-5.15.2/sip/QtWebEngineCore/QtCore/QtCoremod.sip", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/tmp/guix-build-python-pyqtwebengine-5.15.2.drv-0/PyQtWebEngine-5.15.2/sip/QtCore/QtCoremod.sip", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/tmp/guix-build-python-pyqtwebengine-5.15.2.drv-0/PyQtWebEngine-5.15.2/build/bindings/QtCore/QtCoremod.sip", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/lib/python3.8/site-packages/PyQt5/PyQt5/bindings/QtCore/QtCoremod.sip", O_RDONLY) = -1 ENOENT (No such file or directory)
write(2, ".sip-build-real: Unable to find "..., 61.sip-build-real: Unable to find file "QtCore/QtCoremod.sip"


So I'm wondering what the solution to this is. Does SIP 5 & 6 simply not support the Nix style of package management, or is there some simple way for us to make it work?

[-- Attachment #1.2: Type: text/html, Size: 2378 bytes --]

[-- Attachment #2: 0002-gnu-Add-python-pyqt-builder.patch --]
[-- Type: text/x-patch, Size: 1478 bytes --]

From f736357e2232e074dab9032a17deb2ab1eb020dc Mon Sep 17 00:00:00 2001
From: Brendan Tildesley <mail@brendan.scot>
Date: Mon, 22 Feb 2021 22:10:56 +1100
Subject: [PATCH 2/4] gnu: Add python-pyqt-builder.

* gnu/packages/qt.scm (python-pyqt-builder): New variable.
---
 gnu/packages/qt.scm | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/gnu/packages/qt.scm b/gnu/packages/qt.scm
index 38903803be..b9f34f39be 100644
--- a/gnu/packages/qt.scm
+++ b/gnu/packages/qt.scm
@@ -2263,6 +2263,27 @@ itself.")
                        "--sipdir" sip)))))))
     (license (list license:gpl2 license:gpl3)))) ; choice of either license
 
+(define-public python-pyqt-builder
+  (package
+   (name "python-pyqt-builder")
+   (version "1.8.0")
+   (source
+    (origin
+     (method url-fetch)
+     (uri (pypi-uri "PyQt-builder" version))
+     (sha256
+      (base32
+       "0335saz7h1j25gm766ssl3375kw1ak2vs3h764qsxzns3cyzvd9h"))))
+   (build-system python-build-system)
+   (inputs ;; not sure what type of inputs these should be
+    `(("python-toml" ,python-toml)
+      ("python-packaging" ,python-packaging)
+      ("python-sip" ,python-sip)))
+   (home-page "https://www.riverbankcomputing.com/software/pyqt/")
+   (synopsis "The PEP 517 compliant PyQt build system")
+   (description "The PEP 517 compliant PyQt build system")
+   (license (list license:gpl2 license:gpl3))))
+
 (define-public python-qtpy
   (package
     (name "python-qtpy")
-- 
2.30.1


[-- Attachment #3: 0003-pyqt.patch --]
[-- Type: text/x-patch, Size: 3340 bytes --]

From 26b04da35bbdc0bba09f4b4624371e20eb479266 Mon Sep 17 00:00:00 2001
From: Brendan Tildesley <mail@brendan.scot>
Date: Mon, 22 Feb 2021 22:20:40 +1100
Subject: [PATCH 3/4] pyqt

---
 gnu/packages/qt.scm | 33 +++++++++++----------------------
 1 file changed, 11 insertions(+), 22 deletions(-)

diff --git a/gnu/packages/qt.scm b/gnu/packages/qt.scm
index b9f34f39be..33b60a74d1 100644
--- a/gnu/packages/qt.scm
+++ b/gnu/packages/qt.scm
@@ -2019,7 +2019,8 @@ module provides support functions to the automatically generated code.")
                                 "pyqt-public-sip.patch"))))
     (build-system gnu-build-system)
     (native-inputs
-     `(("qtbase" ,qtbase))) ; for qmake
+     `(("qtbase" ,qtbase)
+       ("python-pyqt-builder" ,python-pyqt-builder)))
     (propagated-inputs
      `(("python-sip" ,python-sip)))
     (inputs
@@ -2044,6 +2045,7 @@ module provides support functions to the automatically generated code.")
                   ,@%gnu-build-system-modules)
        #:imported-modules ((guix build python-build-system)
                            ,@%gnu-build-system-modules)
+       #:make-flags (list (string-append "INSTALL_ROOT=" %output))
        #:phases
        (modify-phases %standard-phases
          ;; When building python-pyqtwebengine, <qprinter.h> can not be
@@ -2059,29 +2061,16 @@ module provides support functions to the automatically generated code.")
                #t)))
          (replace 'configure
            (lambda* (#:key inputs outputs #:allow-other-keys)
-             (let* ((out (assoc-ref outputs "out"))
-                    (bin (string-append out "/bin"))
-                    (sip (string-append out "/share/sip"))
-                    (plugins (string-append out "/lib/qt5/plugins"))
-                    (designer (string-append plugins "/designer"))
-                    (qml (string-append plugins "/PyQt5"))
-                    (python (assoc-ref inputs "python"))
-                    (lib (string-append out "/lib/python"
+             (let* ((python (assoc-ref inputs "python"))
+                    (lib (string-append "/lib/python"
                                         (python-version python)
-                                        "/site-packages"))
-                    (stubs (string-append lib "/PyQt5")))
-               (invoke "python" "configure.py"
+                                        "/site-packages")))
+               (invoke "sip-build"
                        "--confirm-license"
-                       "--bindir" bin
-                       "--destdir" lib
-                       "--designer-plugindir" designer
-                       "--qml-plugindir" qml
-                       ; Where to install the PEP 484 Type Hints stub
-                       ; files. Without this the stubs are tried to be
-                       ; installed into the python package's
-                       ; site-package directory, which is read-only.
-                       "--stubsdir" stubs
-                       "--sipdir" sip)))))))
+                       "--no-make"
+                       "--target-dir" lib))))
+         (add-before 'build 'chdir
+           (lambda _ (chdir "build"))))))
     (home-page "https://www.riverbankcomputing.com/software/pyqt/intro")
     (synopsis "Python bindings for Qt")
     (description
-- 
2.30.1


[-- Attachment #4: 0004-pyqtwebengine.patch --]
[-- Type: text/x-patch, Size: 2705 bytes --]

From 4d97677690292acae6406d977353661215ddb23c Mon Sep 17 00:00:00 2001
From: Brendan Tildesley <mail@brendan.scot>
Date: Mon, 22 Feb 2021 22:20:54 +1100
Subject: [PATCH 4/4] pyqtwebengine

---
 gnu/packages/qt.scm | 25 ++++++++++++++-----------
 1 file changed, 14 insertions(+), 11 deletions(-)

diff --git a/gnu/packages/qt.scm b/gnu/packages/qt.scm
index 33b60a74d1..56fa9f18ec 100644
--- a/gnu/packages/qt.scm
+++ b/gnu/packages/qt.scm
@@ -2098,9 +2098,11 @@ contain over 620 classes.")
     (build-system gnu-build-system)
     (native-inputs
      `(("python" ,python)
-       ("python-sip" ,python-sip)
+       ;; ("python-pyqt5-sip" ,python-pyqt5-sip) ;;; XXX do we need it?
+       ("python-pyqt-builder" ,python-pyqt-builder)
        ;; qtbase is required for qmake
-       ("qtbase" ,qtbase)))
+       ("qtbase" ,qtbase)
+       ("strace" ,strace)))
     (inputs
      `(("python" ,python-wrapper)
        ("python-sip" ,python-sip)
@@ -2116,6 +2118,7 @@ contain over 620 classes.")
                   ,@%gnu-build-system-modules)
        #:imported-modules ((guix build python-build-system)
                            ,@%gnu-build-system-modules)
+       #:make-flags (list (string-append "INSTALL_ROOT=" %output))
        #:phases
        (modify-phases %standard-phases
          (replace 'configure
@@ -2128,19 +2131,19 @@ contain over 620 classes.")
                     (lib (string-append out "/lib/python"
                                         (python-version python)
                                         "/site-packages/PyQt5"))
+                    (lib* (string-append "/lib/python"
+                                        (python-version python)
+                                        "/site-packages/PyQt5"))
                     (stubs (string-append lib "/PyQt5")))
 
-               (mkdir-p sipdir)
-               (invoke "python" "configure.py"
-                       "-w"
-                       "--no-dist-info"
-                       "--destdir" lib
-                       "--no-qsci-api"
-                       "--stubsdir" stubs
-                       "--sipdir" sipdir
-                       "--pyqt-sipdir" pyqt-sipdir))))
+               (invoke "strace"
+                       "sip-build"
+                       "--no-make"
+                       "--target-dir" lib*))))
          ;; Because this has a different prefix than python-pyqt then we need
          ;; to make this a namespace of it's own
+         (add-before 'build 'chdir
+           (lambda _ (chdir "build")))
          (add-after 'install 'make-namespace
            (lambda* (#:key inputs outputs #:allow-other-keys)
              (let* ((__init__.py (string-append
-- 
2.30.1


[-- Attachment #5: 0001-gnu-python-sip-Upgrade-to-6.0.1.patch --]
[-- Type: text/x-patch, Size: 2921 bytes --]

From a89fe54bc40fad8b8687e5f586d7f438cd3012ec Mon Sep 17 00:00:00 2001
From: Brendan Tildesley <mail@brendan.scot>
Date: Mon, 22 Feb 2021 21:15:21 +1100
Subject: [PATCH 1/4] gnu: python-sip: Upgrade to 6.0.1.

* gnu/packages/qt.scm (python-sip): Upgrade to 6.0.1.
---
 gnu/packages/qt.scm | 33 +++++++--------------------------
 1 file changed, 7 insertions(+), 26 deletions(-)

diff --git a/gnu/packages/qt.scm b/gnu/packages/qt.scm
index 4fbb157600..38903803be 100644
--- a/gnu/packages/qt.scm
+++ b/gnu/packages/qt.scm
@@ -88,6 +88,7 @@
   #:use-module (gnu packages pulseaudio)
   #:use-module (gnu packages protobuf)
   #:use-module (gnu packages python)
+  #:use-module (gnu packages python-build)
   #:use-module (gnu packages python-xyz)
   #:use-module (gnu packages regex)
   #:use-module (gnu packages ruby)
@@ -1960,7 +1961,7 @@ and binaries removed, and adds modular support for using system libraries.")
 (define-public python-sip
   (package
     (name "python-sip")
-    (version "4.19.24")
+    (version "6.0.1")
     (source
       (origin
         (method url-fetch)
@@ -1970,32 +1971,12 @@ and binaries removed, and adds modular support for using system libraries.")
                                   "/sip-" version ".tar.gz")))
         (sha256
          (base32
-          "1ra15vb5i9gkg2vdvh16cq9x2mmzw1yi3xphxs8q34q1pf83gkgd"))))
-    (build-system gnu-build-system)
+          "11574y8580i239l8cvjjw5fjn409f8bdi2vl4xc6vp5f4hmyjcy9"))))
+    (build-system python-build-system)
     (native-inputs
-     `(("python" ,python-wrapper)))
-    (arguments
-     `(#:tests? #f ; no check target
-       #:imported-modules ((guix build python-build-system)
-                           ,@%gnu-build-system-modules)
-       #:modules ((srfi srfi-1)
-                  ((guix build python-build-system) #:select (python-version))
-                  ,@%gnu-build-system-modules)
-       #:phases
-       (modify-phases %standard-phases
-         (replace 'configure
-           (lambda* (#:key inputs outputs #:allow-other-keys)
-             (let* ((out (assoc-ref outputs "out"))
-                    (bin (string-append out "/bin"))
-                    (include (string-append out "/include"))
-                    (python (assoc-ref inputs "python"))
-                    (lib (string-append out "/lib/python"
-                                        (python-version python)
-                                        "/site-packages")))
-               (invoke "python" "configure.py"
-                       "--bindir" bin
-                       "--destdir" lib
-                       "--incdir" include)))))))
+     `(("python" ,python-wrapper)
+       ("python-toml" ,python-toml)
+       ("python-packaging" ,python-packaging)))
     (home-page "https://www.riverbankcomputing.com/software/sip/intro")
     (synopsis "Python binding creator for C and C++ libraries")
     (description
-- 
2.30.1


^ permalink raw reply related	[flat|nested] 3+ messages in thread

* Re: Upgrading SIP.
  2021-02-22 11:50 Upgrading SIP Brendan Tildesley
@ 2021-02-22 17:50 ` Ludovic Courtès
  2021-02-23  9:29   ` Brendan Tildesley
  0 siblings, 1 reply; 3+ messages in thread
From: Ludovic Courtès @ 2021-02-22 17:50 UTC (permalink / raw)
  To: Brendan Tildesley; +Cc: guix-devel@gnu.org

Hi,

Brendan Tildesley <mail@brendan.scot> skribis:

> Building python-pyqtwebengine, it is unable to find the .sip files needed since it only searches one global path:
>
> openat(AT_FDCWD, "QtCore/QtCoremod.sip", O_RDONLY) = -1 ENOENT (No such file or directory)
> openat(AT_FDCWD, "/tmp/guix-build-python-pyqtwebengine-5.15.2.drv-0/PyQtWebEngine-5.15.2/sip/QtWebEngineCore/QtCore/QtCoremod.sip", O_RDONLY) = -1 ENOENT (No such file or directory)
> openat(AT_FDCWD, "/tmp/guix-build-python-pyqtwebengine-5.15.2.drv-0/PyQtWebEngine-5.15.2/sip/QtCore/QtCoremod.sip", O_RDONLY) = -1 ENOENT (No such file or directory)
> openat(AT_FDCWD, "/tmp/guix-build-python-pyqtwebengine-5.15.2.drv-0/PyQtWebEngine-5.15.2/build/bindings/QtCore/QtCoremod.sip", O_RDONLY) = -1 ENOENT (No such file or directory)
> openat(AT_FDCWD, "/lib/python3.8/site-packages/PyQt5/PyQt5/bindings/QtCore/QtCoremod.sip", O_RDONLY) = -1 ENOENT (No such file or directory)
> write(2, ".sip-build-real: Unable to find "..., 61.sip-build-real: Unable to find file "QtCore/QtCoremod.sip"

Do you know how Qt (or whatever component is involved) searches for .sip
files?

Most of the time, there’s a “search path” typically associated with an
environment variable that we need to set accordingly (like PYTHONPATH,
GUILE_LOAD_PATH, etc.).

Sometimes, that mechanism doesn’t exist upstream so we end up patching
the software to add it.  This is what was done with OBS for instance,
and that patch is now upstream.

HTH!

Ludo’.


^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: Upgrading SIP.
  2021-02-22 17:50 ` Ludovic Courtès
@ 2021-02-23  9:29   ` Brendan Tildesley
  0 siblings, 0 replies; 3+ messages in thread
From: Brendan Tildesley @ 2021-02-23  9:29 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: guix-devel@gnu.org

> On 02/22/2021 6:50 PM Ludovic Courtès <ludo@gnu.org> wrote:
> 
>  
> Hi,
> 
> Brendan Tildesley <mail@brendan.scot> skribis:
> 
> > Building python-pyqtwebengine, it is unable to find the .sip files needed since it only searches one global path:
> >
> > openat(AT_FDCWD, "QtCore/QtCoremod.sip", O_RDONLY) = -1 ENOENT (No such file or directory)
> > openat(AT_FDCWD, "/tmp/guix-build-python-pyqtwebengine-5.15.2.drv-0/PyQtWebEngine-5.15.2/sip/QtWebEngineCore/QtCore/QtCoremod.sip", O_RDONLY) = -1 ENOENT (No such file or directory)
> > openat(AT_FDCWD, "/tmp/guix-build-python-pyqtwebengine-5.15.2.drv-0/PyQtWebEngine-5.15.2/sip/QtCore/QtCoremod.sip", O_RDONLY) = -1 ENOENT (No such file or directory)
> > openat(AT_FDCWD, "/tmp/guix-build-python-pyqtwebengine-5.15.2.drv-0/PyQtWebEngine-5.15.2/build/bindings/QtCore/QtCoremod.sip", O_RDONLY) = -1 ENOENT (No such file or directory)
> > openat(AT_FDCWD, "/lib/python3.8/site-packages/PyQt5/PyQt5/bindings/QtCore/QtCoremod.sip", O_RDONLY) = -1 ENOENT (No such file or directory)
> > write(2, ".sip-build-real: Unable to find "..., 61.sip-build-real: Unable to find file "QtCore/QtCoremod.sip"
> 
> Do you know how Qt (or whatever component is involved) searches for .sip
> files?
> 
> Most of the time, there’s a “search path” typically associated with an
> environment variable that we need to set accordingly (like PYTHONPATH,
> GUILE_LOAD_PATH, etc.).
> 
> Sometimes, that mechanism doesn’t exist upstream so we end up patching
> the software to add it.  This is what was done with OBS for instance,
> and that patch is now upstream.
> 
> HTH!
> 
> Ludo’.

https://www.riverbankcomputing.com/hg/sip/file/tip/sipbuild/builder.py#l273

There is a sip_include_dirs, but the bit that looks for bindings from other packages does so by looking in the target directory specified with --target-dir:

            # Add any bindings from previously installed packages.
            sip_include_dirs.append(
                    os.path.join(project.target_dir,
                            project.get_bindings_dir()))

Besides that it looks in current build directory. 

Currently I'm looking at updating sip and going back to the configure.py script to see if i can get that working first.


^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2021-02-23  9:30 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-02-22 11:50 Upgrading SIP Brendan Tildesley
2021-02-22 17:50 ` Ludovic Courtès
2021-02-23  9:29   ` Brendan Tildesley

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).