unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
* [WIP 0/9] python-build-system set 1: please review
@ 2016-10-10 14:57 Hartmut Goebel
  2016-10-10 14:57 ` [WIP 1/8] gnu: ensure pip and setuptools are installed even for Python 2 Hartmut Goebel
                   ` (9 more replies)
  0 siblings, 10 replies; 11+ messages in thread
From: Hartmut Goebel @ 2016-10-10 14:57 UTC (permalink / raw)
  To: guix-devel

Hi,

this is the first set of patches for the changed python-build-system.

I ask you for review so we can progress in steps.

The whole patch set will be about 50 or 60 patches, where this first set are
the changes of the build system itsefl and the remaining sets will change the
packaging instructions to match the new build-system. Please note that the
packeges will not build with only these first set applied. If you want to test
the complet patch set, please use the branch at
https://gitlab.com/htgoebel/guix/tree/python-build-system.

Concrete I want to know:
* Order of arguments for add-installed-pythonpath and site-packages okay?


Hartmut Goebel (7):
  gnu: ensure pip and setuptools are installed even for Python 2.
  guix: build all Python packages with
    --single-version-externally-managed.
  guix: python-build-system: Add option "#:use-setuptools?" (default
    true).
  guix: python-build-system: Add helpers for getting and setting
    PYTHONPATH.
  guix: python-build-system: Delete .egg-info file created in phase
    check.
  guix: Add lint-checker for packages which should be no inputs at all.
  lint: more packages to probably be a native input.

Marius Bakke (1):
  guix: python-build-system: Import setuptools before calling
    `setup.py'.

 doc/guix.texi                      |  9 +++++
 gnu/packages/python.scm            |  1 +
 guix/build-system/python.scm       |  2 +
 guix/build/python-build-system.scm | 81 +++++++++++++++++++++++++++-----------
 guix/scripts/lint.scm              | 75 ++++++++++++++++++++++++++---------
 tests/lint.scm                     | 34 ++++++++++++++++
 6 files changed, 160 insertions(+), 42 deletions(-)

-- 
2.7.4

^ permalink raw reply	[flat|nested] 11+ messages in thread

* [WIP 1/8] gnu: ensure pip and setuptools are installed even for Python 2.
  2016-10-10 14:57 [WIP 0/9] python-build-system set 1: please review Hartmut Goebel
@ 2016-10-10 14:57 ` Hartmut Goebel
  2016-10-10 14:57 ` [WIP 2/8] guix: build all Python packages with --single-version-externally-managed Hartmut Goebel
                   ` (8 subsequent siblings)
  9 siblings, 0 replies; 11+ messages in thread
From: Hartmut Goebel @ 2016-10-10 14:57 UTC (permalink / raw)
  To: guix-devel

* gnu/packages/python.scm (python-2.7): Add "--with-ensurepip=install" to
  configure-flags.
* doc/guix.texi (Python Modules): Document it.
---
 doc/guix.texi           | 3 +++
 gnu/packages/python.scm | 1 +
 2 files changed, 4 insertions(+)

diff --git a/doc/guix.texi b/doc/guix.texi
index 57821c5..02f7486 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -12250,6 +12250,9 @@ for instance, the module python-dateutil is packaged under the names
 starts with @code{py} (e.g. @code{pytz}), we keep it and prefix it as
 described above.
 
+We currently package Python 2 with @code{setuptools} and @code{pip}
+installed like Python 3.4 has per default.  Thus you don't need to
+specify either of these as an input.
 
 @node Perl Modules
 @subsection Perl Modules
diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index 9b62241..2bdac60 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -156,6 +156,7 @@
        #:configure-flags
        (list "--enable-shared"                    ;allow embedding
              "--with-system-ffi"                  ;build ctypes
+             "--with-ensurepip=install"           ;install pip and setuptools
              (string-append "LDFLAGS=-Wl,-rpath="
                             (assoc-ref %outputs "out") "/lib"))
 
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [WIP 2/8] guix: build all Python packages with --single-version-externally-managed.
  2016-10-10 14:57 [WIP 0/9] python-build-system set 1: please review Hartmut Goebel
  2016-10-10 14:57 ` [WIP 1/8] gnu: ensure pip and setuptools are installed even for Python 2 Hartmut Goebel
@ 2016-10-10 14:57 ` Hartmut Goebel
  2016-10-10 14:57 ` [WIP 3/8] guix: python-build-system: Import setuptools before calling `setup.py' Hartmut Goebel
                   ` (7 subsequent siblings)
  9 siblings, 0 replies; 11+ messages in thread
From: Hartmut Goebel @ 2016-10-10 14:57 UTC (permalink / raw)
  To: guix-devel

This requires setuptools to be installed together with python, which is
the case for Python 3 anyway and which we do for our build of Python 2
(see last commit).

* guix/build/python-build-system.scm (install): Add
  "--single-version-externally-managed" and "--root=/" to params to be
  passed to call-setuppy. Remove thus needless manipulation of
  PYTHONPATH.
---
 guix/build/python-build-system.scm | 21 ++++++---------------
 1 file changed, 6 insertions(+), 15 deletions(-)

diff --git a/guix/build/python-build-system.scm b/guix/build/python-build-system.scm
index e906e60..05a01f3 100644
--- a/guix/build/python-build-system.scm
+++ b/guix/build/python-build-system.scm
@@ -3,6 +3,7 @@
 ;;; Copyright © 2013 Andreas Enge <andreas@enge.fr>
 ;;; Copyright © 2013 Nikita Karetnikov <nikita@karetnikov.org>
 ;;; Copyright © 2015 Mark H Weaver <mhw@netris.org>
+;;; Copyright © 2016 Hartmut Goebel <h.goebel@crazy-compilers.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -64,21 +65,11 @@
                   #:allow-other-keys)
   "Install a given Python package."
   (let* ((out (assoc-ref outputs "out"))
-         (params (append (list (string-append "--prefix=" out))
-                         configure-flags))
-         (python-version (get-python-version (assoc-ref inputs "python")))
-         (old-path (getenv "PYTHONPATH"))
-         (add-path (string-append out "/lib/python" python-version
-                                  "/site-packages/")))
-        ;; create the module installation directory and add it to PYTHONPATH
-        ;; to make setuptools happy
-        (mkdir-p add-path)
-        (setenv "PYTHONPATH"
-                (string-append (if old-path
-                                   (string-append old-path ":")
-                                   "")
-                               add-path))
-        (call-setuppy "install" params)))
+         (params (append (list (string-append "--prefix=" out)
+                               "--single-version-externally-managed"
+                               "--root=/")
+                         configure-flags)))
+    (call-setuppy "install" params)))
 
 (define* (wrap #:key inputs outputs #:allow-other-keys)
   (define (list-of-files dir)
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [WIP 3/8] guix: python-build-system: Import setuptools before calling `setup.py'.
  2016-10-10 14:57 [WIP 0/9] python-build-system set 1: please review Hartmut Goebel
  2016-10-10 14:57 ` [WIP 1/8] gnu: ensure pip and setuptools are installed even for Python 2 Hartmut Goebel
  2016-10-10 14:57 ` [WIP 2/8] guix: build all Python packages with --single-version-externally-managed Hartmut Goebel
@ 2016-10-10 14:57 ` Hartmut Goebel
  2016-10-10 14:57 ` [WIP 4/8] guix: python-build-system: Add option "#:use-setuptools?" (default true) Hartmut Goebel
                   ` (6 subsequent siblings)
  9 siblings, 0 replies; 11+ messages in thread
From: Hartmut Goebel @ 2016-10-10 14:57 UTC (permalink / raw)
  To: guix-devel

From: Marius Bakke <mbakke@fastmail.com>

This is needed for packages using "distutils" instead of "setuptools" since
the former does not understand the "--single-version-externally-managed"
flag. Also export __file__ since it will be unset when setup.py is called from
python "exec".

* guix/build/python-build-system.scm (call-setuppy): extend "python setup.py"
  call to import setuptools, export __file__, and call setup.py from
  setuptools python environment.

Co-Authored-By: Hartmut Goebel <h.goebel@crazy-compilers.com>
---
 guix/build/python-build-system.scm | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/guix/build/python-build-system.scm b/guix/build/python-build-system.scm
index 05a01f3..50a094d 100644
--- a/guix/build/python-build-system.scm
+++ b/guix/build/python-build-system.scm
@@ -36,13 +36,23 @@
 ;;
 ;; Code:
 
+(define setuptools-shim
+  ;; Run setup.py with "setuptools" being imported, which will patch
+  ;; "distutils". This is needed for packages using "distutils" instead of
+  ;; "setuptools" since the former does not understand the
+  ;; "--single-version-externally-managed" flag.
+  ;; Python code taken from pip 8.1.2 pip/utils/setuptools_build.py
+  (string-append
+   "import setuptools, tokenize;__file__='setup.py';"
+   "exec(compile(getattr(tokenize, 'open', open)(__file__).read()"
+   ".replace('\\r\\n', '\\n'), __file__, 'exec'))"))
 
 (define (call-setuppy command params)
   (if (file-exists? "setup.py")
       (begin
          (format #t "running \"python setup.py\" with command ~s and parameters ~s~%"
                 command params)
-         (zero? (apply system* "python" "setup.py" command params)))
+         (zero? (apply system* "python" "-c" setuptools-shim command params)))
       (error "no setup.py found")))
 
 (define* (build #:rest empty)
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [WIP 4/8] guix: python-build-system: Add option "#:use-setuptools?" (default true).
  2016-10-10 14:57 [WIP 0/9] python-build-system set 1: please review Hartmut Goebel
                   ` (2 preceding siblings ...)
  2016-10-10 14:57 ` [WIP 3/8] guix: python-build-system: Import setuptools before calling `setup.py' Hartmut Goebel
@ 2016-10-10 14:57 ` Hartmut Goebel
  2016-10-10 14:57 ` [WIP 5/8] guix: python-build-system: Add helpers for getting and setting PYTHONPATH Hartmut Goebel
                   ` (5 subsequent siblings)
  9 siblings, 0 replies; 11+ messages in thread
From: Hartmut Goebel @ 2016-10-10 14:57 UTC (permalink / raw)
  To: guix-devel

TODO: Documentation, test-case.

* guix/build-system/python.scm (python-build): New keyword argument
  "#:use-setuptools?", defaulting to #t.
* guix/build/python-build-system.scm (call-setup-py): New positional
  parameter "use-setuptools?". If false, do not use the shim-wrapper
  for addin setuptools. (build, check): accept keyword-
  parameter, and pass to call-setuppy. (install): same; if
  "use-setuptools?" is false, do not use options "--root" and
  "--single-version-externally-managed" for setup.py.
* doc/guix.texi (Build Systems): Document it.
---
 doc/guix.texi                      |  5 +++++
 guix/build-system/python.scm       |  2 ++
 guix/build/python-build-system.scm | 28 +++++++++++++++++-----------
 3 files changed, 24 insertions(+), 11 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 02f7486..92a827a 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -3092,6 +3092,11 @@ the @code{#:python} parameter.  This is a useful way to force a package
 to be built for a specific version of the Python interpreter, which
 might be necessary if the package is only compatible with a single
 interpreter version.
+
+By default guix calls @code{setup.py} under control of
+@code{setuptools}, much like @command{pip} does.  Some packages are not
+compatible with setuptools (and pip), thus you can disable this by
+setting the @code{#:use-setuptools} parameter to @code{#f}.
 @end defvr
 
 @defvr {Scheme Variable} perl-build-system
diff --git a/guix/build-system/python.scm b/guix/build-system/python.scm
index adeceb4..d4d3d28 100644
--- a/guix/build-system/python.scm
+++ b/guix/build-system/python.scm
@@ -177,6 +177,7 @@ pre-defined variants."
                        #:key
                        (tests? #t)
                        (test-target "test")
+                       (use-setuptools? #t)
                        (configure-flags ''())
                        (phases '(@ (guix build python-build-system)
                                    %standard-phases))
@@ -204,6 +205,7 @@ provides a 'setup.py' file as its build system."
                      #:system ,system
                      #:test-target ,test-target
                      #:tests? ,tests?
+                     #:use-setuptools? ,use-setuptools?
                      #:phases ,phases
                      #:outputs %outputs
                      #:search-paths ',(map search-path-specification->sexp
diff --git a/guix/build/python-build-system.scm b/guix/build/python-build-system.scm
index 50a094d..038fa97 100644
--- a/guix/build/python-build-system.scm
+++ b/guix/build/python-build-system.scm
@@ -47,22 +47,25 @@
    "exec(compile(getattr(tokenize, 'open', open)(__file__).read()"
    ".replace('\\r\\n', '\\n'), __file__, 'exec'))"))
 
-(define (call-setuppy command params)
+(define (call-setuppy command params use-setuptools?)
   (if (file-exists? "setup.py")
       (begin
          (format #t "running \"python setup.py\" with command ~s and parameters ~s~%"
                 command params)
-         (zero? (apply system* "python" "-c" setuptools-shim command params)))
+         (if use-setuptools?
+             (zero? (apply system* "python" "-c" setuptools-shim
+                           command params))
+             (zero? (apply system* "python" "./setup.py" command params))))
       (error "no setup.py found")))
 
-(define* (build #:rest empty)
+(define* (build #:key use-setuptools? #:allow-other-keys)
   "Build a given Python package."
-  (call-setuppy "build" '()))
+  (call-setuppy "build" '() use-setuptools?))
 
-(define* (check #:key tests? test-target #:allow-other-keys)
+(define* (check #:key tests? test-target use-setuptools? #:allow-other-keys)
   "Run the test suite of a given Python package."
   (if tests?
-    (call-setuppy test-target '())
+    (call-setuppy test-target '() use-setuptools?)
     #t))
 
 (define (get-python-version python)
@@ -71,15 +74,18 @@
          (major+minor (take components 2)))
     (string-join major+minor ".")))
 
-(define* (install #:key outputs inputs (configure-flags '())
+(define* (install #:key outputs inputs (configure-flags '()) use-setuptools?
                   #:allow-other-keys)
   "Install a given Python package."
   (let* ((out (assoc-ref outputs "out"))
-         (params (append (list (string-append "--prefix=" out)
-                               "--single-version-externally-managed"
-                               "--root=/")
+         (params (append (list (string-append "--prefix=" out))
+                         (if use-setuptools?
+                             ;; distutils does not accept these flags
+                             (list "--single-version-externally-managed"
+                                    "--root=/")
+                             '())
                          configure-flags)))
-    (call-setuppy "install" params)))
+    (call-setuppy "install" params use-setuptools?)))
 
 (define* (wrap #:key inputs outputs #:allow-other-keys)
   (define (list-of-files dir)
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [WIP 5/8] guix: python-build-system: Add helpers for getting and setting PYTHONPATH.
  2016-10-10 14:57 [WIP 0/9] python-build-system set 1: please review Hartmut Goebel
                   ` (3 preceding siblings ...)
  2016-10-10 14:57 ` [WIP 4/8] guix: python-build-system: Add option "#:use-setuptools?" (default true) Hartmut Goebel
@ 2016-10-10 14:57 ` Hartmut Goebel
  2016-10-10 14:57 ` [WIP 6/8] guix: python-build-system: Delete .egg-info file created in phase check Hartmut Goebel
                   ` (4 subsequent siblings)
  9 siblings, 0 replies; 11+ messages in thread
From: Hartmut Goebel @ 2016-10-10 14:57 UTC (permalink / raw)
  To: guix-devel

* guix/build/python-build-system.scm (add-installed-pythonpath,
  site-packages): New exported procedures.
---
 guix/build/python-build-system.scm | 22 +++++++++++++++++++++-
 1 file changed, 21 insertions(+), 1 deletion(-)

diff --git a/guix/build/python-build-system.scm b/guix/build/python-build-system.scm
index 038fa97..1950b3b 100644
--- a/guix/build/python-build-system.scm
+++ b/guix/build/python-build-system.scm
@@ -28,7 +28,9 @@
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-26)
   #:export (%standard-phases
-            python-build))
+            python-build
+            add-installed-pythonpath
+            site-packages))
 
 ;; Commentary:
 ;;
@@ -74,6 +76,24 @@
          (major+minor (take components 2)))
     (string-join major+minor ".")))
 
+(define (site-packages outputs inputs)
+  "Return the path of the current output's Python site-package."
+  (let* ((out (assoc-ref outputs "out"))
+         (python (assoc-ref inputs "python")))
+    (string-append out "/lib/python"
+                   (get-python-version python)
+                   "/site-packages/")))
+
+(define (add-installed-pythonpath outputs inputs)
+  "Prepend the Python site-package of OUTPUT to PYTHONPATH.  This is useful
+when running checks after installing the package."
+  (let ((old-path (getenv "PYTHONPATH"))
+        (add-path (site-packages outputs inputs)))
+    (setenv "PYTHONPATH"
+            (string-append add-path
+                           (if old-path (string-append ":" old-path) "")))
+    #t))
+
 (define* (install #:key outputs inputs (configure-flags '()) use-setuptools?
                   #:allow-other-keys)
   "Install a given Python package."
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [WIP 6/8] guix: python-build-system: Delete .egg-info file created in phase check.
  2016-10-10 14:57 [WIP 0/9] python-build-system set 1: please review Hartmut Goebel
                   ` (4 preceding siblings ...)
  2016-10-10 14:57 ` [WIP 5/8] guix: python-build-system: Add helpers for getting and setting PYTHONPATH Hartmut Goebel
@ 2016-10-10 14:57 ` Hartmut Goebel
  2016-10-10 14:57 ` [WIP 7/8] guix: Add lint-checker for packages which should be no inputs at all Hartmut Goebel
                   ` (3 subsequent siblings)
  9 siblings, 0 replies; 11+ messages in thread
From: Hartmut Goebel @ 2016-10-10 14:57 UTC (permalink / raw)
  To: guix-devel

* guix/build/python-build-system.scm (check): Delete .egg-info dirs
  which did not exist prior to calling setup.py but afterwards.
---
 guix/build/python-build-system.scm | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/guix/build/python-build-system.scm b/guix/build/python-build-system.scm
index 1950b3b..e85df40 100644
--- a/guix/build/python-build-system.scm
+++ b/guix/build/python-build-system.scm
@@ -67,7 +67,15 @@
 (define* (check #:key tests? test-target use-setuptools? #:allow-other-keys)
   "Run the test suite of a given Python package."
   (if tests?
-    (call-setuppy test-target '() use-setuptools?)
+      ;; Running `setup.py test` creates an additional .egg-info directory in
+      ;; build/lib in some cases, e.g. if the source is in a sub-directory
+      ;; (given with `package_dir`). This will by copied to the output, too,
+      ;; so we need to remove.
+      (let ((before (find-files "build" "\\.egg-info$" #:directories? #t)))
+        (call-setuppy test-target '() use-setuptools?)
+        (let* ((after (find-files "build" "\\.egg-info$" #:directories? #t))
+               (inter (lset-difference eqv? after before)))
+          (for-each delete-file-recursively inter)))
     #t))
 
 (define (get-python-version python)
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [WIP 7/8] guix: Add lint-checker for packages which should be no inputs at all.
  2016-10-10 14:57 [WIP 0/9] python-build-system set 1: please review Hartmut Goebel
                   ` (5 preceding siblings ...)
  2016-10-10 14:57 ` [WIP 6/8] guix: python-build-system: Delete .egg-info file created in phase check Hartmut Goebel
@ 2016-10-10 14:57 ` Hartmut Goebel
  2016-10-10 14:57 ` [WIP 8/8] lint: more packages to probably be a native input Hartmut Goebel
                   ` (2 subsequent siblings)
  9 siblings, 0 replies; 11+ messages in thread
From: Hartmut Goebel @ 2016-10-10 14:57 UTC (permalink / raw)
  To: guix-devel

Also refactor some common code into a new function.

Examples for these pacakges are python(2)-setuptools and python(2)-pip, which
are installed together with python itself.

* guix/scripts/lint.scm (warn-if-package-has-input): New procedure.
  (check-inputs-should-be-native package): Use it; rename and clean-up
  variables. (check-inputs-should-not-be-an-input-at-all): New procedure.
  (%checkers) Add it.
* doc/guix.texi (Python Modules): Document it.
* tests/lint.scm: ("inputs: python-setuptools should not be an input at all
  (input)", "inputs: python-setuptools should not be an input at all
  (native-input)" "inputs: python-setuptools should not be an input at all
  (propagated-input)"): Add tests.
---
 doc/guix.texi         |  3 ++-
 guix/scripts/lint.scm | 63 ++++++++++++++++++++++++++++++++++++---------------
 tests/lint.scm        | 34 +++++++++++++++++++++++++++
 3 files changed, 81 insertions(+), 19 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 92a827a..fa5b89e 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -12257,7 +12257,8 @@ described above.
 
 We currently package Python 2 with @code{setuptools} and @code{pip}
 installed like Python 3.4 has per default.  Thus you don't need to
-specify either of these as an input.
+specify either of these as an input.  @command{guix lint} will warn you
+if you do.
 
 @node Perl Modules
 @subsection Perl Modules
diff --git a/guix/scripts/lint.scm b/guix/scripts/lint.scm
index b3ec6d6..042c679 100644
--- a/guix/scripts/lint.scm
+++ b/guix/scripts/lint.scm
@@ -4,6 +4,7 @@
 ;;; Copyright © 2013, 2014, 2015, 2016 Ludovic Courtès <ludo@gnu.org>
 ;;; Copyright © 2015, 2016 Mathieu Lirzin <mthl@gnu.org>
 ;;; Copyright © 2016 Danny Milosavljevic <dannym+a@scratchpost.org>
+;;; Copyright © 2016 Hartmut Goebel <h.goebel@crazy-compilers.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -59,6 +60,7 @@
   #:export (guix-lint
             check-description-style
             check-inputs-should-be-native
+            check-inputs-should-not-be-an-input-at-all
             check-patch-file-names
             check-synopsis-style
             check-derivation
@@ -213,34 +215,55 @@ by two spaces; possible infraction~p at ~{~a~^, ~}")
                       (format #f (_ "invalid description: ~s") description)
                       'description))))
 
+(define (warn-if-package-has-input linted inputs-to-check input-names message)
+  ;; Emit a warning MESSAGE if some of the inputs named in INPUT-NAMES are
+  ;; contained in INPUTS-TO-CHECK, which are assumed to be inputs of package
+  ;; LINTED.
+  (match inputs-to-check
+    (((labels packages . outputs) ...)
+     (for-each (lambda (package output)
+                 (when (package? package)
+                   (let ((input (string-append
+                                 (package-name package)
+                                 (if (> (length output) 0)
+                                     (string-append ":" (car output))
+                                     ""))))
+                     (when (member input input-names)
+                       (emit-warning linted
+                                     (format #f (_ message) input)
+                                     'inputs-to-check)))))
+               packages outputs))))
+
 (define (check-inputs-should-be-native package)
   ;; Emit a warning if some inputs of PACKAGE are likely to belong to its
   ;; native inputs.
-  (let ((linted package)
+  (let ((message "'~a' should probably be a native input")
         (inputs (package-inputs package))
-        (native-inputs
+        (input-names
           '("pkg-config"
             "extra-cmake-modules"
             "glib:bin"
             "intltool"
             "itstool"
             "qttools")))
-    (match inputs
-      (((labels packages . outputs) ...)
-       (for-each (lambda (package output)
-                   (when (package? package)
-                     (let ((input (string-append
-                                   (package-name package)
-                                   (if (> (length output) 0)
-                                       (string-append ":" (car output))
-                                       ""))))
-                       (when (member input native-inputs)
-                         (emit-warning linted
-                                       (format #f (_ "'~a' should probably \
-be a native input")
-                                               input)
-                                       'inputs)))))
-                 packages outputs)))))
+    (warn-if-package-has-input package inputs input-names message)))
+
+(define (check-inputs-should-not-be-an-input-at-all package)
+  ;; Emit a warning if some inputs of PACKAGE are likely to should not be
+  ;; an input at all.
+  (let ((message "'~a' should probably not be an input at all")
+        (inputs (package-inputs package))
+        (input-names
+          '("python-setuptools"
+            "python2-setuptools"
+            "python-pip"
+            "python2-pip")))
+    (warn-if-package-has-input package (package-inputs package)
+                               input-names message)
+    (warn-if-package-has-input package (package-native-inputs package)
+                               input-names message)
+    (warn-if-package-has-input package (package-propagated-inputs package)
+                               input-names message)))
 
 (define (package-name-regexp package)
   "Return a regexp that matches PACKAGE's name as a word at the beginning of a
@@ -810,6 +833,10 @@ them for PACKAGE."
      (description "Identify inputs that should be native inputs")
      (check       check-inputs-should-be-native))
    (lint-checker
+     (name        'inputs-should-not-be-input)
+     (description "Identify inputs that should be inputs at all")
+     (check       check-inputs-should-not-be-an-input-at-all))
+   (lint-checker
      (name        'patch-file-names)
      (description "Validate file names and availability of patches")
      (check       check-patch-file-names))
diff --git a/tests/lint.scm b/tests/lint.scm
index d692b42..148e162 100644
--- a/tests/lint.scm
+++ b/tests/lint.scm
@@ -3,6 +3,7 @@
 ;;; Copyright © 2014, 2015, 2016 Eric Bavier <bavier@member.fsf.org>
 ;;; Copyright © 2014, 2015, 2016 Ludovic Courtès <ludo@gnu.org>
 ;;; Copyright © 2015, 2016 Mathieu Lirzin <mthl@gnu.org>
+;;; Copyright © 2016 Hartmut Goebel <h.goebel@crazy-compilers.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -33,6 +34,7 @@
   #:use-module (gnu packages)
   #:use-module (gnu packages glib)
   #:use-module (gnu packages pkg-config)
+  #:use-module (gnu packages python)
   #:use-module (web server)
   #:use-module (web server http)
   #:use-module (web response)
@@ -346,6 +348,38 @@ string) on HTTP requests."
           (check-inputs-should-be-native pkg)))
           "'glib:bin' should probably be a native input")))
 
+(test-assert
+    "inputs: python-setuptools should not be an input at all (input)"
+  (->bool
+   (string-contains
+     (with-warnings
+       (let ((pkg (dummy-package "x"
+                    (inputs `(("python-setuptools" ,python-setuptools))))))
+         (check-inputs-should-not-be-an-input-at-all pkg)))
+         "'python-setuptools' should probably not be an input at all")))
+
+(test-assert
+    "inputs: python-setuptools should not be an input at all (native-input)"
+  (->bool
+   (string-contains
+     (with-warnings
+       (let ((pkg (dummy-package "x"
+                    (native-inputs
+                     `(("python-setuptools" ,python-setuptools))))))
+         (check-inputs-should-not-be-an-input-at-all pkg)))
+         "'python-setuptools' should probably not be an input at all")))
+
+(test-assert
+    "inputs: python-setuptools should not be an input at all (propagated-input)"
+  (->bool
+   (string-contains
+     (with-warnings
+       (let ((pkg (dummy-package "x"
+                    (propagated-inputs
+                     `(("python-setuptools" ,python-setuptools))))))
+         (check-inputs-should-not-be-an-input-at-all pkg)))
+         "'python-setuptools' should probably not be an input at all")))
+
 (test-assert "patches: file names"
   (->bool
    (string-contains
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [WIP 8/8] lint: more packages to probably be a native input.
  2016-10-10 14:57 [WIP 0/9] python-build-system set 1: please review Hartmut Goebel
                   ` (6 preceding siblings ...)
  2016-10-10 14:57 ` [WIP 7/8] guix: Add lint-checker for packages which should be no inputs at all Hartmut Goebel
@ 2016-10-10 14:57 ` Hartmut Goebel
  2016-10-11  8:08 ` [WIP 0/9] python-build-system set 1: please review Hartmut Goebel
  2016-10-11  8:37 ` Hartmut Goebel
  9 siblings, 0 replies; 11+ messages in thread
From: Hartmut Goebel @ 2016-10-10 14:57 UTC (permalink / raw)
  To: guix-devel

* guix/scripts/lint.scm (check-inputs-should-be-native package): Add python
  packages which are typically used for testing or for building the
  documentation.
---
 guix/scripts/lint.scm | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/guix/scripts/lint.scm b/guix/scripts/lint.scm
index 042c679..cd630c1 100644
--- a/guix/scripts/lint.scm
+++ b/guix/scripts/lint.scm
@@ -245,7 +245,17 @@ by two spaces; possible infraction~p at ~{~a~^, ~}")
             "glib:bin"
             "intltool"
             "itstool"
-            "qttools")))
+            "qttools"
+            "python-coverage" "python2-coverage"
+            "python-cython" "python2-cython"
+            "python-docutils" "python2-docutils"
+            "python-mock" "python2-mock"
+            "python-nose" "python2-nose"
+            "python-pbr" "python2-pbr"
+            "python-pytest" "python2-pytest"
+            "python-pytest-cov" "python2-pytest-cov"
+            "python-setuptools-scm" "python2-setuptools-scm"
+            "python-sphinx" "python2-sphinx")))
     (warn-if-package-has-input package inputs input-names message)))
 
 (define (check-inputs-should-not-be-an-input-at-all package)
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 11+ messages in thread

* Re: [WIP 0/9] python-build-system set 1: please review
  2016-10-10 14:57 [WIP 0/9] python-build-system set 1: please review Hartmut Goebel
                   ` (7 preceding siblings ...)
  2016-10-10 14:57 ` [WIP 8/8] lint: more packages to probably be a native input Hartmut Goebel
@ 2016-10-11  8:08 ` Hartmut Goebel
  2016-10-11  8:37 ` Hartmut Goebel
  9 siblings, 0 replies; 11+ messages in thread
From: Hartmut Goebel @ 2016-10-11  8:08 UTC (permalink / raw)
  To: guix-devel

Hi,
> this is the first set of patches for the changed python-build-system.

I was asked about the big plan on these commits.


The original proposal to this changes can be found at
<http://lists.gnu.org/archive/html/guix-devel/2016-09/msg02021.html>.
And this change sets implement

*A1) Change the Python 2 to include pip and setuptools (build using
--with-ensurepip=install).
**B) Install Python packages using **--single-version-externally-managed
C) Strip **python-setuptools and ***python2-setuptools* from all
native-inputs defined in packages.*

And additionally - since it showed up to be required:
*NEW: D) Change inputs into propagated resp. native-inputs where required.*


About the planed patch sets:

Set 1 are the changes to the build system

Set 2 will be the removal of obviously now unused inputs, namely setuptools

    These will be straight forward.

Set 3 will be the straight forward changes to inputs (change to
propagated resp.native)

Set 4 will be the fixes and clean ups required to some packages after
the these changes.

    I'm not sure whether I should split set 3 and set 4 nor have I
    decided yet whether set 3 should only contain those packages which
    will not be changed in set 4. But this makes updating the patches
    quite complicated (even when using git rebase).

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [WIP 0/9] python-build-system set 1: please review
  2016-10-10 14:57 [WIP 0/9] python-build-system set 1: please review Hartmut Goebel
                   ` (8 preceding siblings ...)
  2016-10-11  8:08 ` [WIP 0/9] python-build-system set 1: please review Hartmut Goebel
@ 2016-10-11  8:37 ` Hartmut Goebel
  9 siblings, 0 replies; 11+ messages in thread
From: Hartmut Goebel @ 2016-10-11  8:37 UTC (permalink / raw)
  To: guix-devel

Am 10.10.2016 um 16:57 schrieb Hartmut Goebel:
> If you want to test
> the complet patch set, please use the branch at
> https://gitlab.com/htgoebel/guix/tree/python-build-system.

Please note that this branch *may* change frequently if I update clean
up commits.

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2016-10-11  8:37 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2016-10-10 14:57 [WIP 0/9] python-build-system set 1: please review Hartmut Goebel
2016-10-10 14:57 ` [WIP 1/8] gnu: ensure pip and setuptools are installed even for Python 2 Hartmut Goebel
2016-10-10 14:57 ` [WIP 2/8] guix: build all Python packages with --single-version-externally-managed Hartmut Goebel
2016-10-10 14:57 ` [WIP 3/8] guix: python-build-system: Import setuptools before calling `setup.py' Hartmut Goebel
2016-10-10 14:57 ` [WIP 4/8] guix: python-build-system: Add option "#:use-setuptools?" (default true) Hartmut Goebel
2016-10-10 14:57 ` [WIP 5/8] guix: python-build-system: Add helpers for getting and setting PYTHONPATH Hartmut Goebel
2016-10-10 14:57 ` [WIP 6/8] guix: python-build-system: Delete .egg-info file created in phase check Hartmut Goebel
2016-10-10 14:57 ` [WIP 7/8] guix: Add lint-checker for packages which should be no inputs at all Hartmut Goebel
2016-10-10 14:57 ` [WIP 8/8] lint: more packages to probably be a native input Hartmut Goebel
2016-10-11  8:08 ` [WIP 0/9] python-build-system set 1: please review Hartmut Goebel
2016-10-11  8:37 ` Hartmut Goebel

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).